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Description 

1. Field of the Invention 

5 [0001 ] The present invention relates to object extraction method and apparatus for extracting a specific image from 
color images or black-and-white images. Further present invention relates to color-image recognition method and ap- 
paratus for determining whether the thus-extracted specific image is an image from given images. In practice, such a 
specific-image extraction is performed as follows: Data representing a specific image is extracted from data represent- 
ing color images or black-and-white images. Further, data representing a specific image part is extracted from data 

io representing a specific color image or a black-and-white image. 

[0002] Each of the color-image recognition method and apparatus uses the thus -extracted specific image and then 
determines whether the specific image corresponds to an image included in given images ! which are, for example, 
images constituting an image database, in particular, color-image database. 

[0003] In an example, such a technique may be applied to works in a police office. The police office may have a 
database of criminals' face photographs. When a montage picture (composite picture) is given by a witness, the police 
officer attempts to determine whether the image of thus-given picture is an image included in the database. 
[0004] In another example, an intelligent building may use such a technique. A microcomputer is provided inside a 
door of a room of a firm and the microcomputer uses a database of employees' face photographs. If a person comes 
in front of the door the microcomputer uses a CCD camera to take image data of the person's face and, by using the 
thus-taken image data, the microcomputer attempts to determine whether the same image data is included in the 
database. If the microcomputer has succeeded in determining the same image data is in the employees' face database, 
the microcomputer opens the door for the person and if not, the microcomputer does not open it. Thus, no real key is 
needed for the door. 

[0005] In another example, such a technique may be used to prevent bank notes and securities from being replicated. 
25 a duplicator may have image data of bank notes and securities and if an operator sets an original image on the duplicator 
to obtain a copy thereof, the duplicator checks if data of the thus-set original image is included in the image data of 
bank notes and securities. 
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2. Prior Art 



[0006] Apparatuses processing color images such as a color duplicator, a color printer, a color scanner, a color- 
image communication apparatus, and so forth are expected to be further developed in the future. It has become easier 
to handle the color images by hardware development, such as cost reductions and capacity increase of various sorts 
of memories, data-communication cost reduction and so forth. However, since data representing the color images has 
35 a huge data amount (for example, 96 Mbytes for an image of the A3 size) it is impossible to handle the color images 
similariy to handling data representing two-tone images. 

[0007] In particular in handling image data, a complicates process such as image recognition (such as recognition 
of a specific image, OCR or the like) requires a huge amount of data to be processed. Therefore, it is further difficult 
to realize the image recognition for the color images. 
40 [0008] Various methods have been proposed as an object extraction method for extracting an object to be recognized. 
In an example method, the method extracts black-continuity rectangles from a relevant image and compares values 
representing the thus-extracted rectangles with predetermined threshold values. The method classifies the rectangles 
into rectangles representing characters and rectangles representing line drawings. See Japanese Laid-Open Patent 
Application No.55-162177. However, this method neither classifies the thus-classified rectangles of line drawings into 
rectangles representing horizontal ruled-lines, nor has a capability to extract a specific image even if the relevant image 
is present but is in a rotated form. 

[0009] As an apparatus and method for identifying a specific color image, a color-image processing apparatus (see 
Japanese Laid-Open Patent Application No.4-54681) was proposed, which apparatus extracts hue and distribution of 
the hue from a given region of an original image and thus detects whether or not a specific image is present. However, 
the above-described apparatus uses only 32 points to be sampled, the number of points to be sampled being a few in 
order to reduce a data amount to be processed. Such using of a few sampled points may not result in obtaining a 
sufficient amount of information required to extract characteristics of a relevant image. Therefore, it is not possible to 
obtain highly accurate image identification performance. 

[001 0] Further a method was proposed for determining whether or not a specific color appears and then extracting 
image parts having the same characteristics (see Japanese Laid Open Patent Application No.4-1 80348). Each of 
patterns constituting an image has a particular stretch in a color space. The above-mentioned method uses such 
specific stretches, in the color space, of the patterns to identify a certain image part as being a specific image part, the 
certain image part having specific stretches in the color space the same as those of the specific image part. However, 
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the method cannot distinguish image parts having the same stretch in the color space but a different color distribution 

inside the stretch. Thus, if a certain image part has a stretch in the color space, which stretch is the same as the stretch 

of a specific image part, but a color distribution inside the stretch is different from that of the stretch of the specific 

image part, the method may erroneously identify the certain image part as being the specific image part. 

[0011] Document EP-A-0 294 497 discloses a postage stamp identifying apparatus extracting color characteristic 

patterns of the stamp as a feature vector from a stamp, and the detected color characteristic pattern of the stamp is 

compared with each of standard color characteristic patterns prepared before-hand for known stamps. 

[0012] An object of the present invention is to provide a color image recognition method and apparatus, each of 

which extracts an image region corresponding to an object from an input image with high accuracy. 

[0013] Another object of the present invention is to provide color-image recognition methods and apparatus, each 

of which method and apparatus extracts an object from an input image and performs a color-image recognition operation 

on the object. Each of the color-image recognition method and apparatus obtains a sufficient amount of information 

required for the color-image recognition operation so that high-accuracy color-image recognition can be achieved with 

a data amount to be processed being compressed. 

[0014] Another object of the present invention is to provide color-image recognition methods and apparatus, each 
of which method and apparatus uses a table-conversion manner to convert relevant data, instead of performing a 
calculation operation, so as to compress the amount of the relevant data. Thus, it is possible to achieve an effective 
data-amount compression. 

[0015] Another object of the present invention is to provide color-image recognition methods and apparatus, each 
of which method and apparatus suppresses an adverse effect of noise image information. The noise image information 
may be included in relevant information as a result of, for example, an image scanner reading in information of not 
only a relevant image printed on a relevant sheet but also of another image which is printed on another sheet subsequent 
to the relevant sheet. The image of the subsequent sheet may be read by the scanner through the relevant sheet if it 
is thin. Each of the method and apparatus performs a vector quantizing operation only on the relevant image information. 
Thus, image-recognition capability and data processing speed are improved. 

[0016] Another object of the present invention is to provide color-image recognition methods and apparatus, each 
of which method and apparatus determines a similarity between information of an input image and information of a 
predetermined code book information. If the thus-determined similarity is higher than a predetermined threshold value, 
no vector-quantizing value is assigned to the information of the input image. Thus, image-recognition capability and 
data processing speed are improved. 

[0017] Color image recognition methods and apparatus are provided according to the invention as set out in claims 
1 and 12, respectively. 

[0018] To achieve the above-mentioned objects, color image recognition methods and apparatus according to the 
present invention determine a circumscribing rectangle for a black continuous image part using a two-tone image 
signal. A height and a horizontal length of the rectangle are then compared with threshold values. If an operator specifies 
that an object to be extracted from an input image has a rectangular shape, whether or not the relevant image part is 
possibly one which is obtained as a result of rotating the object is determined. If it is determined that the image part is 
possibly one which is obtained as a result of rotating the object, under an assumption that the image part also has a 
rectangular shape, lengths of sides of the image part are compared with threshold values. If it is determined that the 
image part is not one which is obtained as a resutt of rotating the object, the height and horizontal length of the cir- 
cumscribing rectangle is the threshold values. Thus, it is determined whether or not the image part is identical to the 
object. 

[0019] Further, each of color image recognition methods and apparatus according to the present invention divides 
an input image into small regions. An RGB (red, green and blue) chromaticity histogram is produced for each of the 
small regions. Each of the chromaticity histograms of the small regions is compared with reference ones, named code 
book information, previously produced and stored. Each of the thus-stored reference chromaticity histograms has an 
identification number. As a result of the comparison, the identification number of the histogram having the highest 
similarity to that of the small region among those constituting the input image is given to the small region. A histogram 
of the identification numbers thus given to the small regions constituting the input image is produced. The thus-produced 
histogram is compared with reference ones, named dictionary information, previously produced and stored. As a result 
of the comparison, a reference histogram of identification numbers among the thus-stored reference ones is selected, 
the thus-selected one having the highest similarity to that of the input image. Each of the above-mentioned reference 
histograms of identification numbers corresponds to a respective one of a plurality of objects. As a result of the above- 
mentioned selection, one of the plurality of objects is determined to be identical to the input image. Thus, the input 
image is recognized. 

[0020] Other objects and further features of the present invention will become more apparent from the following 
detailed description when read in conjunction with the accompanying drawings. 
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FIG.1 shows a block diagram of a first embodiment of a color image recognition method and apparatus according 
to the present invention; 

FIG.2 shows an operation flow chart of the above-mentioned embodiment shown in FIG.1; 
FIG. 3 illustrates a method of recognizing a location of a circumscribing rectangle and a location of an image part 
circumscribed by the circumscribing rectangle, the method being performed by a rectangle extracting unit in the 
embodiment shown in FIG.1; 

FIG.4 illustrates a method of determining whether or not the relevant image part is possibly one obtained as a 
result of rotating an object, the method being performed by a rotation determining unit in the embodiment shown 
in FIG.1; 

FIG.5 show a block diagrams of second, third and fourth embodiments of the color image recognition method and 
apparatus according to the present invention; 

FIGS. 6 and 7 show block diagrams of a fifth embodiment of the color image recognition method and apparatus 
according to the present invention; 

FIG. 8 illustrates a method of dividing an input image part into small regions, the method being performed by a 
15 mesh dividing unit in the embodiment shown in FIG. 7; 

FIGS. 9 and 10 illustrate a vector quantizing operation performed by a vector quantizing executinq unit shown in 
FIG.7; 

FIGS. 11 A and 11 B shows an operation flow of the embodiments shown in FIGS. 6 and 7; 
FIG.1 2 illustrates a scope defining operation performed through a step S203 shown in FIG.11B; 
20 FIG. 13 shows an object identifying operation performed through a step S205 shown in FIG.11 B; 

FIG.1 4 shows a block diagram of a sixth embodiment of color image recognition method and apparatus according 
to the present invention; 

FIG. 15 shows an operation flow of the embodiment shown in FIG. 14; 

FIG.1 6 illustrates a pixel selecting operation which is a feature of a seventh embodiment a color image recognition 
25 method and apparatus according to the present invention; 

FIG. 1 7 shows an operation flow of ninth and tenth embodiments of a color image recognition method and apparatus 
according to the present invention; 

FIG.1 8 shows a block diagram an eleventh embodiment of a color image recognition method and apparatus ac- 
cording to the present invention; 

FIG.1 9 illustrates a conversion compressing operation performed by a characteristic extracting unit 37B using a 
conversion compressing table 50 shown in FiG.18; and 

FIG.20 illustrates a vector quantizing operation performed by a vector quantizing unit 38 shown in FIG.18. 
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[0021] With reference to FIG.1, an object extraction system in a first embodiment of the color image recognition 
methods and apparatus according to the present invention will now be described. In general, the system is effective 
in the use of an image having a rectangular outline as the object, an image part corresponding to the object being 
extracted through the system. That is, the system is produced generally based on an assumption that said image part 
has a rectangular shape. Accordingly, names of units in the system and associated expressions may be those based 
on the object having a rectangular outline. However, the system can use a figure other than a rectangle as the object. 
A two-tone image signal generating unit 1 generates two-tone image signal using an input image. A circumscribing 
rectangle extracting unit 2 extracts circumscribing rectangles from the thus-generated two-tone image signal. 
[0022] Hereinafter, to extract a circumscribing rectangle means to obtain a circumscribing rectangle described below. 
The circumscribing rectangle is a rectangle, two of four sides of which extend along a scan-line direction in which the 
input image is scanned so as to be read in. The circumscribing rectangle is extracted for a black-continuity image part 
present in an image represented by the two-tone image signal. The black-continuity image parts consists of black 
pixels constituting a black continuous region in the image. The circumscribing rectangle is a rectangle which circum- 
scribes the relevant black-continuity image part. 

[0023] A rectangle memory 3 stores rectangle data representing the thus-extracted circumscribing rectangle. A can- 
didate-rectangle determining unit 4 which uses the thus-stored circumscribing rectangle data and then compares a 
horizontal length and a height of each of the circumscribing rectangles with first preset threshold values (representing 
first preset extents for the object). The first preset extents are an extent in height and an extent in horizontal dimension. 
Thus, the determining unit 4 determines whether or not each circumscribing rectangle might be possibly correspond 
to an object which is a black-continuity image part to be extracted from the image. Further, the candidate-rectangle 
determining unit 4 determines whether or not a shape of the object is a rectangle. 

[0024] If the candidate-rectangle determining unit 4 has determined the circumscribing rectangle to possibly be the 
object, a candidate-rectangle memory 5 stores candidate-rectangle data representing the thus-determined circum- 
scribing rectangle which will be referred to as a candidate rectangle. 

[0025] A rotation determining unit 6, if the candidate-rectangle determining unit 4 has determined the shape of the 
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object is a rectangle, further checks the candidate rectangle. The rotation determining unit 6 determines whether or 
not the relevant candidate rectangle may possibly include the object therein and the relevant image part is possibly 
one which was obtained as a result of rotating the object. 

[0026] A side-length measuring unit 7 measures a length of a longer side and a length of the shorter side of the 
relevant black-continuity image part. An object determining unit 8 compares the thus-measured lengths with second 
preset threshold values (representing second preset extent for the object) so as to determine whether or not the can- 
didate rectangle corresponds to the object. The second preset extents are an extent in height and an extent in horizontal 
dimension. The second preset extents for the object are narrower than the above-mentioned first preset extents for 
the object, respectively. By using the first and second preset extents for the object, it is possible to step by step determine 
the possibility of the relevant image part corresponding to the object. Thus, using the first extents for the object, if the 
relevant image part actually corresponds to the object but the image part is one which is obtained as a result of rotating 
the object, failing to extract the image part can be prevented since the first extentsfor the object are relatively wide 
respectively. Using the relatively narrow second extents for the object, it is possible to accurately determine the cor- 
respondence between the relevant image part and the object. An object- rectangle memory 9 stores data representing 
the outline of the relevant image part if the candidate rectangle has been determined to correspond to the object through 
the object determining unit 8. A control unit 10 controls the units 2, 4, 6, 7 and 8 in the system. 
[0027] With reference to FIG.2, an operation flow charts of the object extraction system shown in FIG.1 will now be 
described. A step S101 generates the two-tone image signal using the input image through the two-tone image signal 
generating unit 1. Specifically, for example, the input image may be input in a form of a multi-tone image black-and- 
white signal, and the unit 1 may have a predetermined threshold value. The unit 1 uses the predetermined threshold . 
value and thus obtains the two-tone image signal. 

[0028] That is, if a value of a pixel represented by the multi-tone image signal is equal or less than the threshold 
value : the value is determined to be a black pixel and if the value of the pixel is greater than the threshold value, the 
pixel is determined to be a white pixel. 

[0029] A step S102 extracts the circumscribing rectangle from the two-tone image signal through the rectangle ex- 
tracting unit 2. Any one of methods proposed by the present applicant through Japanese Laid-Open Patent Applications 
Nos.3-341889, 4-267313, 4-160866 and so forth may be used for the circumscribing-rectangle extraction. 
[0030] FIG.3 shows a circumscribing rectangle 202, as an example, extracted from the input image 201. The cir- 
cumscribing rectangle 202 is that extracted for the black-continuity image part 203. According to the first embodiment, 
in the step S1 02, coordinates (x s , y s ), (x e , y e ), (x s , y e ) and (x e , y s ) of four vertexes of the rectangle 202 are extracted. 
Further, simultaneously to the vertex-coordinate extraction, coordinates (x u , y s ), (x e , y r ), (Xg, y,) and (x b , y e ) for points 
at which the circumscribing rectangle comes into contact with the black-continuity image part are extracted. 
[0031] The above description has been made based on the image part 203 being a rectangle. However if the image 
part 203 is not a rectangle, a similar method may be applied. That is, if the image part 203 is any figure, a number of 
points are present at which the image part comes into contact with the circumscribing rectangle. The step S102 may 
obtain coordinates of the number of points and data of the thus-obtained coordinates may be used in the subsequent 
steps similarly to those obtained in the case where the image part 203 is a rectangle. An essential point is that if the 
object is present in the input image part, it should be accurately extracted. Thus, which figure the relevant image part 
203 is, is not an essential point, but it is essential that comparison between reference data associated with the object 
and data associated with the relevant image part 203 should be accurately performed. Such accurate comparison is 
achieved as a result of ensuring that how the reference data associated with the object was obtained is the same as 
how the data, to be compared with the reference data, associated with the image part 203 is obtained. The above- 
described method for obtaining information indicating the position of the outline of the circumscribing rectangles and 
that of the image parts is a method what is simple yet enables an effective position recognition. 
[0032] A step S103 determines whether or not the height and the horizontal length of the circumscribing rectangle 
are in the first preset extents for the object through the candidate-rectangle determining unit 4. That is, the step S1 03 
determines whether or not the height is present between a relevant preset maximum allowable limit and a minimum 
allowable limit and similarly determines whether or not the horizontal length is present between a relevant preset 
maximum allowable limit and a minimum allowable limit. If at least one of the height and horizontal length is not present 
within the relevant extent, a step S1 1 6 determines that the relevant image part does not correspond to the object. One 
of the above-mentioned methods proposed by the present applicant may be used for the dimension-using object- 
candidate determining operation performed by the step S103. A step S104 determines whether or not the shape of 
the object is a rectangle through the candidate-rectangle determining unit 4. Whether or not the shape of the object is 
a rectangle is, for example, previously input by an operator and the thus-input data is preset in the candidate- rectangle 
determining unit 4 before the above-described object extracting operation is actually performed. It is noted that deter- 
mining in S1 04 whether or not the shape of the object is a rectangle is performed based on information from the outside 
of the system, but not based on a result of the system's checking the actually input image signal. 
[0033] If the step S104 has determined that the object shape is a rectangle, a step S1 05 determines whether or not 
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a rotation angle <s present between the relevant image part and the scan-line direction through the rotation determining 
unit 6. FIG.4 shows a candidate rectangle 301 for an image part 302 as an example. The rotation determination is 
performed in the following manner: If the image part 302 is a rectangle as shown in FIG.4, it is determined whether or 
not a triangle A is congruent with a triangle B and also it is determined whether or not a triangle C is congruent with a 
triangle D, shown in FIG.4. If at least one of the above-mentioned two congruities is not admitted in the rotation deter- 
mination, it is determined that the relevant image part is not one which was obtained as a result of rotating the object 
If each of the above-mentioned two congruities is admitted in the rotation determination, it is determined that the 
relevant image part is possibly one which was obtained as a result of rotating the object. Further, if the above-mentioned 
coordinates (x y s ), (x e , y e ) ; (x s , y e ) and <x e , y s ) of thefourvertexes of the rectangle 202 are coincide with the coordinates 
(x u . y s ) (x e . y r )> y,) and (x b , y e ) of the points at which the circumscribing rectangle comes into contact with the black- 
continuity image part are extracted, respectively, it is determined that the relevant image part is not one which was 
obtained as a result of rotating the object. " ' " 

[0< !foL FUrthen 3 ' ength ° f 3 dia9 ° nal " ne 01 is ^P 3 ^ a length of the other diagonal line D2 of the image 
part 302 as shown in FIG.4. Each length of the lengths of the diagonal lines D1 and D2 is calculated from the coordinates 
sucn as (x u , y s ), (x e , y r ), (x s , y ( ) and fx b , y e ), shown in FIG.3, of the rectangle data obtained for the image part 302 by 

he step S102. As a result of the above-mentioned length comparison, if a difference between the lengths of the diagonal 
lines D1 and D2 is relatively long, it is determined that the relevant image part has a shape of a diamond and not of a 
rectangle, thus the image part 302 is not one obtained as a result of rotating the object. Such a method for determining 

he rotation of the object using the congruency of the triangles is a method simple but enabling effective rotation de- 
^ U termination. 

[0035] If the step S1 05 has determined that the image part is possibly one obtained as a result of rotating the object 
a step S1 06, through the side-length measuring unit 7, calculates a length s1 of a first side and a length s2 of a second 
side adjoining the first side of the image part 302 and shown in FIG.4. The length calculation is performed using the 

C °-o!?o u S ' SUCh 33 ( * u ' Vs) ' (Xe ' Yr) ' (Xs ' Yl) and (x "' shown in FIG ' 3 ' of tne rectangle data obtained for the image 
part 302 by the step S1 02. The step S1 06 then determines the thus-calculated lengths as lengths of a short side and 
a long side of the relevant image part. However, if the step S1 04 has determined that the shape of the object is not a 
rectangle, the step S 1 07 determines that the height and horizontal length of the circumscribing rectangle are the lengths 
of the short side and long side of the image part. Further, if the step S105 has determined that the image part is not 
one obta.ned as a result of rotating the object, the step S107 determines that the height and horizontal length of the 
circumscribing rectangle are the lengths of the short side and long side of the image part. 

[0036] A step S1 08 determines whether or not the thus-determined lengths of the short side and long side of the 
image part are in the second preset extents for the object through the object determining unit 8. That is the step S1 08 
determines whether or not the short-side length is present between a relevant preset maximum allowable limit and a 
minimum allowable limit and similarly determines whether or not the long-side length is present between a relevant 
preset maximum allowable limit and a minimum allowable limit. If at least one of the short-side and long-side lengths 
is not present within the relevant extent, a step S1 1 6 determines that the relevant image part does not correspond to 
the object. If the short-side and long-side lengths are present within the relevant extents, respectively the step S11 5 
determines that the relevant image part corresponds to the object. The above-described operation flow shown in FIG 
2 will be performed on all the black-continuity image parts included in the input image in the form of the two-tone image 
signal. a 

[0037] The determination performed in the step S104 is based on, for examole, the operator's input as mentioned 
above. Therefore, whether or not the two-tone image signal obtained in the step S101 corresponds to a rectangle 
depends on cases. That is, as one case, it is assumed that the object to be extracted is a rectangular business card 
which is made of a small white paper sheet having black letters printed thereon. In such a case, the operator may 
determine the object as being a rectangle and thus inputs information, indicating the object as being a rectangle into 
he candidate-rectangle determining unit 4. Then, if the identical business card is used as an image part to be checked 
through the object extraction system shown in FIG.1 , the two-tone image signal generating unit 1 generates the signal 
representing the black letters printed on the white papersheet, but does not generate that representing the white paper 
sheet, in the step S1 01 . As a result, the step S102 extracts the rectangles of the black letters 
[0038] In one typical rectangle extraction method, one rectangle is extracted for each black letter and thus a com- 
bmation of the rectangles for the black letters is obtained through the step S101 , a general overall outline the above- 
mentioned combination does not ordinary have a rectangular shape. As a result, the shape of the image part is deter- 
mined as not being a rectangle. However, the step S104 determines the object as being a rectangle according to the 
above-mentioned operator's input. In such a case, the step S104 determines YES and the step S105 determines NO 
This is because the above-mentioned triangle congruity described with reference to FIG.4 is not admitted in the rotation 
determination, as a result of the relevant image part (302 in FIG.4) being the combination of the rectangles and the 
combination having general overall outline not being a rectangular shape as mentioned above. Whether or not the 
relevant image part is present as one resulting from being rotated, the triangle congruity is not admitted, unless the 
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relevant image part has a general overall outline being a rectangular shape. In fact, if the relevant image part has not 
a rectangular overall outline, a shape of each of areas present defined by a part of a border of the circumscribing 
rectangle and a part of the outline of the relevant image part does not constitute a triangle. In such a case, the step 
S 107 is executed after the step S105. Thus, the step S107 is performed correctly for the relevant image part not a 

5 rectangle in this case although the step S104 determines the object as being the rectangle. In such a case, the above- 
mentioned second preset extents for the object were predetermined so as to match the black letters of the business 
card. A longer one of a height and a horizontal length of the combination of the rectangles for black letters of the 
business card is used as the long-side length used in the step S108 as the reference data. Further, a shorter one of 
the height and horizontal length is used as the short-side length used therein. Thus, the relevant image part is correctly 

10 determined as the object in the step S115 unless the relevant image part is one resulting from rotating the object. 
[0039] It is noted that, normally, each letter of such black letters of the business card may be determined as being 
an individual small black-continuity image part and thus result in an individual small circumscribing rectangle through 
the rectangle extracting unit 2. The thus-obtained small circumscribing rectangles for the black letters of the business 
card may be integrated into a single rectangle, the integration being performed on circumscribing rectangles which 

is have mutual distances each distance being shorter than a predetermined threshold value. Thus, the above-mentioned 
combination of the rectangles for black letters of the business card can be obtained. The above-mentioned predeter- 
mined threshold value may be determined to match geometric arrangement of letters printed in such a business card. 
Such an integrating step may be inserted in the rectangle extraction step S 102 in a case where such letters printed on 
a business card are used as the above-mentioned object. 

20 [0040] Further, it is possible to insert, between the two-tone image signal generating unit 1 and the rectangle ex- 
tracting unit 2, an image compressing unit for converting each group of 8 pixels of the input image in the form of the 
two-tone image signal into one pixel. Thus, an image signal obtained as a result of compressing the two-tone image 
signal is obtained and the thus-obtained image signal is used to extract the circumscribing rectangles. 
[0041] An object extraction system in a second embodiment of object extraction method and apparatus according 

25 to the present invention will now be described. Difference between the object extraction system in the second embod- 
iment and that in the first embodiment is that the system in the second embodiment uses a color-image signal as the 
input signal while the system in the first embodiment uses the multi-tone black-and-white-image signal as the input 
signal as described above. Except for this point, the two systems are identical. Specifically, the two-tone image signal 
generating unit 1 , shown in FIG.1 , in the system in the second embodiment operates as follows: 

30 [0042] It is assumed that the input color-image signal carries three color component reflectance values R (red), G 
(green) and B (blue) for each pixel. The reflectance values are large if the pixel is white and the reflectance values are 
small if the pixel is black. The two-tone image signal generating unit 1 derives a lightness value I using the reflectance 
values R, G and B through the following equation: 

35 

L=R+G+B 

The two-tone image signal generating unit 1 in the second embodiment determines each pixel as being black if 

40 

L<Th1 , 

and determines the pixel as being white if 

45 

L>Th1. 

The sign Th1 represents a predetermined threshold value. Thus, the generating unit 1 determines a pixel having a 
lightness lower than the value Th1 as being black and determines the pixel having a lightness higher than the value 
so as being white. The thus-determined color, black or white, is used as being a value carried by the two-tone signal for 
each pixel. Thus, the two-tone signal is generated. 

[0043] This method is effective for a case in which a sheet having an original image to be checked is placed in a 
white background. That is, if the system is applied to a machine such as a duplicator and the duplicator has a glass 
on which the sheet, for example the business card having color letters printed thereon in this case, is to be placed. 
55 The duplicator further has a plate which has a white surface and is placed on the glass, so that the white surface faces 
the glass, after the sheet is placed on the glass. The above-mentioned glass has, for example, a size of the A3 size 
and the white surface has a size larger than the A3 size accordingly. The duplicator scans the sheet and the white 
surface of the plate, both having been placed on the glass. As a result, the pixels corresponding to the white surface 
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[0045] The two-tone image signal generating unit 1 in the third embodiment determines each pixel as being black if 



and determines the pixel as being white if 



L>Th2, 



L<Th2. 



The sign Th2 represents a predetermined threshold value. Thus the aeneratinn „ n it 1 a*t a rm^ ■ , u 

mmm 

h! nivl , eXamP ' e ' determined as being black (L>Th2). Therefore, in all the relevant pLels onlv 

ESS Funh?/ * thiS ifS ^ ^ SyStemS in thS SeC ° nd and third embodiments are identical. 

imem has r 6 r ^V r0m ^ ° f ^ SySt6mS °' the SeC ° nd and third embodiments, the system in the fourth embod- 
TeZ s Z^ZZ FK3 5 EST? 3 ^ ^ S ^ 3 ' m «4 A «d a twoteTmage 
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two-tone image signal memory A, 12 and the two-tone image signal obtained from the above-mentioned second op- 
eration is stored in the two-tone image signal memory B, 13. 

[0051] The rectangle extracting unit 2 extracts the circumscribing rectangles using the two-tone image signal stored 
in the memory 12, the thus-extracted circumscribing rectangles being referred to as first rectangles. Further, the rec- 
tangle extracting unit 2 extracts the circumscribing rectangles also using the two-tone image signal stored in the memory 
13, the thus extracted circumscribing rectangles being referred to as second rectangles. The candidate-rectangle de- 
termining unit 4 performs the step S103 shown in FIG.2 so as to determine for each rectangle of the first and second 
rectangles whether or not the rectangle has dimensions matching the first preset extents for the object, respectively. 
As a result, a rectangle among the above-mentioned first rectangles may be determined to have dimensions included 
in the first preset extents respectively for the object and it will be referred to as a first rectangle. Similarly, a rectangle 
among the above-mentioned second rectangles may be determined to have dimensions included in the first preset 
extents respectively for the object and it will be referred to as a second rectangle. 

[0052] The candidate-rectangle determining unit 4 then determines whether or not the first rectangle completely 
geometrically includes the second rectangle or vice versa or no such inclusion is present in the rectangles. If such 
inclusion is present, only one rectangle which completely includes the other rectangle is stored in the rectangle memory 
3 and the other rectangle completely included in the rectangle, having been stored in the memory 3 as mentioned 
above, is removed from further determination. If, for example, the above-mentioned second rectangle is geometrically 
completely included in the first rectangle, only the first rectangle is stored in the memory 3 and will be further determined. 
Such geometrical determination can eliminate redundancy which results from performing further determination on the 
two rectangles, one geometrically completely includes the other. 

[0053] The above-described operation will now be described using the above-mentioned example using the business 
card as the input image to be checked in the object extraction system. If the duplicator having the plate with the white 
surface is used as the plate to be placed on the glass, the white surface of the plate is determined as being white in 
the two-tone image generating unit 1 1 in the first operation according to the method of the system in the second em- 
bodiment. Further, the white background of the card, placed on the glass at the same time, is determined as being 
white and the color letters is determined as being black. As a result, the first circumscribing rectangles obtained from 
the two-tone image signal is a rectangle representing the color letters. On the other hand, in the second operation 
according to the method of the system in the third embodiment, the two-tone image generating unit 11 provides the 
two-tone image signal in which the white surface of the plate and the white background of the business card are 
determined as being black (L>Th2). As a result, the second circumscribing rectangle obtained from the two-tone image 
signal is a rectangle representing the white surface of the plate which has dimensions, for example, of a size larger 
than the A3 size, as mentioned above. On the other hand, the first rectangle having the dimensions of the business 
card, for example, 9 cm x 5.5 cm. The thus-obtained second rectangle has very large area in comparison to the first 
rectangle. If the object to be extracted is a business card such as that placed on the glass of the duplicator as mentioned 
above, the above-mentioned first preset extents (in dimensions) for the object thus are those including the dimensions 
of the business card, respectively, accordingly. Since the dimensions of the A3 size are very large in comparison to 
those of the business card, the second rectangle will not be determined to have dimensions included in the first preset 
extents respectively for the object in the step S1 03. As a result, only the first rectangle will be determined to have the 
dimensions included in the first preset extents respectively for the object, and thus will be stored in the rectangle 
memory 3, and further determination will be performed on the thus-stored rectangle. 

[0054] In this example, if the operator does not use the plate and thus the plate is not placed on the glass, the 
duplicator scans the open space present on and above the glass and thus it is determined as being black according 
to the method of the second embodiment. Further, the color letters printed on the business card are also determined 
as being black. Thus, the first circumscribing rectangles is those representing the open space and the color letters of 
the business card. On the other hand, in the two-tone image signal obtained in accordance with the method according 
to the third embodiment, the open space is determined as being white and the white background of the business card 
is determined as being black. Thus, the second circumscribing rectangle is that representing the business card. Thus, 
the three circumscribing rectangles are obtained, first one for the open space on and above the glass, second one for 
the color letters of the card and third one for the business card. As mentioned above, the candidate-rectangle deter- 
mining unit 4 determines that each of the second one for the color letters of the card and the third one for the business 
card among the three circumscribing rectangles are included in the first preset extents respectively for the object. This 
is because the circumscribing rectangle for the open space has dimensions similar to the white surface of the plate 
and very large in comparison to the those of the business card. Further, the circumscribing rectangle for the color letters 
of the card has dimensions relatively close to those of the business card since, generally speaking, an overall outline 
of a set of letters printed on the business card has a major area in the entire area of the business card. As mentioned 
above, the candidate-rectangle determining unit 4 then determines the geometric inclusion of the thus-obtained two 
rectangles, the above-mentioned second one for the letters and third one for the card. Since the card geometrically 
completely included the letters, the third circumscribing rectangle for the card is selected to be stored in the rectangle 
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memory 3, and further determination will be performed on the thus-stored rectangle 

above f JlVoK 0 ? Pre f 8 ' eXtents f dimensions, long-side dimension and short-side dimension as mentioned 

kind thelf r JS ^ 6 St6P 3108 ° f FiG 2 38 thS r6ference data P referab 'V include ^0 kinds of extents !rst 
kind thereof for the u mage part representing the objeot obtained by the method according to the second embodiment 

eloZTJZ U°: the f ima9S repreSemin9 the ° bj6Ct ° btained b * the ^hod accord ^ng to theXd 
embodiment. These two kinds of extents are, in the above-described example, one kind thereof for the business ca d 
anc the other land thereof for the letters printed for the business card. Thus the step 31 08 can accural y p rfoTmTe 

IZXZ^^J^FP 3n 8PPr0Priate ° f S6COnd Preset 6 * entS of the above-menC" st 
kind the eof and .econd kind thereof. That ,s, if the circumscribing rectangle obtained as a result of the method ac 

cording to the second embodiment is provided to the step S108, the first kind of the second preset e« S the 
the Srd r ' ^ "* " ^ drCUmSCribin 9 rectan 9<e stained as a result o, the method a^olg o 
usei in ttTSJSS." ^ 10 ^ StSP S108 ' SeC ° nd kind ° f the second — * «* objects 
[0056] Thus, the above-described object extraction system in the fourth embodiment can flexibly process the imaae 
data or venous cases, for example, no, only the above-mentioned case in which the plate with the whS surface is 
used to be p aced on the glass on which a sheet, such as a business card was placed on, but also h above JZSSS 

fnTsec^^ 

Hh X Urthen il f f° P ° SSible that 3 Pie ° es ° f P hot °Sraph is used as the above-mentioned object to be extracted 
through th6 present object extraction system, instead of using a business card for the same puiose as described 

the oTotuZT' h P ' eC H e ° f Ph0t0Qraph haS 3 Whi,S b ° rder P ° rtion i(1 the P^eV <* a picture pomon Ordinaril 

owe tan E^JT iTTi ^ ^ *" ° f ^ ^ that is ' tbe P icture P° rt '° n has a lightness 

tower than that of the white border portion. Further, an outline of each portion of the picture portion and border pS 

mlnH ! I f l P ln SUCh 3 ° aSe ' " the tw °- ,one ima 9 e si 9 nal is generated due to the above-described 
sciS^ 

fhTth h reCtan ? le ln S1 ° 2 ' ,f lhe *»°-tone image signal is generated due to the above-described method accordina to 

Picture ooln Z th ° *5 b ° rder P ° rti ° n 9«ica«,y c ° m P |e tely includes the rectangular outline of the 

picture portion, onty the crcumscnb.ng rectangle extracted for the border portion is stored in the rectangle memory 3 
?no«f Cl ™ mscnbln 9 Wangle extracted for the picture portion is not stored therein 9 * ' 

meZ .^SZT^TSf r ° bjeCt t eXtraCti ° n afld rec °9 niti - in a fifth embodiment of object extraction 

described FunSnl ofTho S meth ° d a " d aPParatUS aCC ° rdin 9 t0 the P resent ^vention will now be 

described. Functions of the object extracting unit 22 includes those of the two-tone image signal generating unit 11 

mi ni q n ur a 6 9 : a s nS t T em r es , A and B - 12 and 13 * rec,angie ex,racting un * *■ rnL-snzsi 

F G 5 Full Tp t t f° 9 ? mem0fy 5 ' Side "' eng,h meaSUring Unit 7 ' and ob i ect determining unit 8 shown in 
FIG.5. Further, the object-rectangle memory 9 is identical to that shown in FIG 5 

^olor A ir a T s S!f a 9 ? 4 rK° rmS 3 V6Ct0r qU3ntiZing ° Perati ° n ' deSCribed later ' ° n »• c °'°™ a 9* signal. 

oo £ 9 I T 3 S ' gnal Wh ' Ch IS ' npUt 10 the s y stem shown in FIG. 6 and processed in the object extracts 
I^ref 1 :ToFIG e 5 ° Ut,i : e °! h' ' T* ^ * *• ^-mentioned obje^ 252 

oToIrfnlinn h^t !• tSd and St ° red in the ^^angle memory 9. Values obtained as a result 

25 A oZ? qUam ' Zlng ° Perati0n 6n thS color - ima 9 e s i9" a ' a 'e stored in a vector quantized value memory 

I nd in rec ;9 nlzln 9 un 't 26 uses the values, stored in the vector quantized value memory 25 as mentioned above 
Ln.H h T : epresentin 9 the outlins o' the relevant image part, stored in object-rectangle memory g as men 
loned above with reference to FIGS.1 and 5. Using the values and data, the object ecognizing un^26 seTects vZs 
" ? VeCt0rqUanliZ6d values ~™»y « -ing the data stored in the object ^5S^^ 

ec s Th^ L TT m and reC ° gniZe the tb ^-extracted one as the object, includes a P Sy of 

nrn^c . ' eCOgn ' ZingUnit26detemine ^ 

slTd f'r 896 Part : S re ' eVant ' 38 dSSCribed latSr ™ S reC °9 nizin 9 °P era t'°" h perfonmed by corner g the vS 
selected from among the memory 25 as mentioned above, with values, which values are previously produce Sna™ 

SSJ? f T d ' ater ThUS ' the Unit 26 detemlines Whether or not the image da a ^S^S^SSZ 
un!, s sCnt^G T 6 "^ ' C ° rreSP ° ndS l ° ^ ab °— ttoned ° b ^t- A coi. unit 27 ovJrall contls thSer 
[0061] With reference to FIG.7 and FIGS.11 A and 11 B, details of the object extraction and recognition system in the 
fifth embodiment will now be described. As mentioned above, the object extracting unit 22 and ZobiSS^2 
memory 9 have functions identical to those of the object extraction system shown InTHG^n add ton to'th f net o 
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identical to those of the system shown in FiG.5, the object extracting unit 22 has a function to provide a trigger signal. 
The above-mentioned trigger signal indicates timing at which the object extracting unit 22 receives data s which data 
is included in data carried by the above-mentioned color-image signal, and represents a certain one of the image parts. 
The certain one of the image part is an image part for which the object extracting unit 22 has extracted the circumscribing 

5 rectangle and also has determined that the thus extracted circumscribing rectangle as corresponding to the object as 
described above through the step S11 5 shown in FIG.2. After the object extracting unit 22 receives the above-mentioned 
data : the object-rectangle memory 9 provides scope data consisting of the data representing the outline of the relevant 
image part in the memory 9. The vector quantizing unit 24 consists of a vector quantizing processing unit 35 and a 
memory 39 named code book, described later, shown in FIG. 7. The object recognizing unit 26 consists of an object 

10 recognizing processing unit 41 and a memory 44 named dictionary, described later, shown in FIG.7. The above-men- 
tioned trigger signal and scope data are provided to the vector recognizing processing unit 41 . 
[0062] The vector quantizing processing unit 35 includes a mesh dividing unit 36, a characteristic extracting unit 37, 
and a vector quantizing executing unit 38. The mesh dividing unit 36, in a step S201 shown in FIG.11B, divides the 
color image carried by the color-image signal (consisting of R, G, and B signals) into small regions. Such dividing is 

15 performed as if a mesh (grid) is placed on the image. The characteristic extracting unit 37, in a step S202A of FIG. 
11 B, obtains an amount representing characteristics of each region of the thus-divided small regions. As described 
later, the amount representing characteristics consists of a chromaticity histogram. The vector quantizing executing 
unit 38, in a step S202B of FIG. 11 B, compares the thus -extracted amounts with information previously produced and 
stored in the code book 39. By the comparison, the unit 38 performs the vector quantizing operation in the step S202B. 

20 Data is obtained as a result of the vector quantizing operation performed on the amounts representing characteristics 
of the small regions of the color image data. The thus-obtained data is stored in the vector quantized value memory 
25 and may be referred to as "vector quantized values', hereinafter. All of data carried by the color-image signal input 
to the object extraction and recognition system shown in FIG. 6 is processed by the vector quantizing unit 24. 
[0063] How to perform the vector quantizing operation will now be described. FIG. 8 shows a comer of the above- 

25 mentioned color image. A finer grid shown in FIG.3 indicates pixels of pixels constituting the relevant color images. A 
coarser grid indicates the small regions divided by the mesh dividing unit 36. In the example, each small region of the 
small regions consists of 64 pixels x 64 pixels. (In FIG.8, indication of 64 pixels x 64 pixels for the small region is omitted 
of, but 28 pixels and 28 pixels are indicated therefor, for the sake of simplification of the drawing.) The mesh dividing 
unit 36 divides the small regions as described above and obtains image data of the 64 pixels x 64 pixels for each small 

30 region to be processed by the characteristic extracting unit 37. However, it is possible to reduce the number of pixels 
of the image data to be processed by the unit 37. Specifically, for example, as shown in FIG. 16, pixels are sampled 
from the 64 pixels x 64 pixels so that 7 pixels, along each of the vertical and horizontal directions, are left between 
each two pixels to be sampled. Thus, the number of pixels to be sampled is a number of 1/8 x 1/8 the 64 x 64, that is, 
64-1/8 x 64-1/8. for each small region. Image data of the thus-reduced number of pixels, that is, the thus compressed 

35 image data may be processed by the characteristic extracting unit 37. 

[0064] The characteristic extracting unit 37 obtains chromaticities Pr and Pg by the following equations (1) and (2) 
for each pixel using color image values R, G and B, carried by the above-mentioned color-image signal and each value 
of R, G and B consisting of data of 8 bit: 



40 

Pr=256-R/(R+G+B) (1), 



and 

45 



Pg=256-G/(R+G+B) (2). 



[0065] The thus-obtained chromaticities Pr and Pg of the pixels have only information of tone of colors and does not 
have information dependent of a matter, for example, as to how strong a sheet having the color image printed thereon 
is illuminated, the color image being scanned to obtain the color-image signal. Thus, accurate object recognition can 
be achieved by using the chromaticities Pr and Pg without being adversely affected by, for example, the above-men- 
tioned illumination. If object (image) recognition is made using image data depending on the illumination, adverse 
affection may occur in the following situation, for example: 

[0066] It is assumed that the image data is obtained as a result of scanning a book placed on a glass of a duplicator, 
while certain two pages of the book are opened and face the glass, the image scanning being made through the glass. 
In the scanning, first light is illuminated on the opened pages. If the book is thick, that is. the book has large number 
of pages, and the above-mentioned two pages are opened on the glass, parts of the two pages adjoining with each 
other at a binding part of the book may not come into contact with the glass and thus may be left without sufficiently 
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F!G9 01 sh O wr, h l riZ ° ntal iS thUS ^ aXiS ° f the chromaticit y dimensions. The vertical axis of the histogram 401 of 
STL ?*? "° v / ' ,for ' 7 ' atf0 " 402 shown in FIG 9 ^ information stored in the code book 39 shown in FIG 7 The 

mwmmm 



12 



EP 0 657 839 B1 



DCj= 2 (Hi-Cji) 2 • • • (4) . 
i*0 



The above sign 'Hi* is referred to the appearance frequency at the chromaticity dimension Y of the histogram 401 . The 
above sign 'Cjf is referred to the appearance frequency at the chromaticity dimension ? of the histogram Cj included 
in the code book information 402. As shown in the above equation (4), each difference (Hi-Cj) may be squared and 
summed for i = 0 through 511 . Thus, the above-mentioned distance DCj is obtained. A method of obtaining the above- 
mentioned distance between the relevant histogram 401 and each one of the histograms of the code book information 
402 is not limited to this method. Various methods are known and may be used for the same purpose. Using the 
distances thus-obtained for the histograms constituting the code book information 402, the vector quantizing executing 
unit 38 determines a chromaticity histogram Cj from among those constituting the code book information 402. The 
histogram to be determined as mentioned above is a chromaticity histogram Cj having the minimum distance from the 
histogram 401 obtained from the relevant small region. The vector quantizing executing unit 38 stores the number j of 
the thus-determined histogram Cj in the vector quantized memory 40. The number j thus stored for each small region 
is referred to as a 'vector quantized values' or a 'VQ value', the small region being one among the small regions 
constituting the color image represented by the input color-image signal. The VQ values are stored in the vector quan- 
tized value memory 25 shown in FIG.7. 

[0074] The object recognizing processing unit 41 includes a vector quantized value histogram producing unit 42 and 
a matching unit 43. The histogram producing unit 42 uses the above-mentioned vector quantized values of ones among 
the above-mentioned small regions and thus produces a histogram. FIG. 10 shows an example of the input color image 
which is divided as the 9 x 11 small regions through the mesh dividing unit 36 of FIG.7. A numeral shown in each small 
region in FIG. 10 is the relevant VQ value. The histogram of the vector quantized values will now be described. If, for 
example, the histogram is produced for all of the small regions shown in FIG. 10, VQ 0 =0, VQ.,=8 : VQ 2 =15, VQ 3 =7, 
VQ 4 =9, VQ 5 =14, VQ 6 =9 : VQ^.. VQ 8 =12, VQg=5, ... For example, 'VQ^IS' is obtained since the numeral '2' is present 
in FIG. 10 '15' times. That is, an appearance frequency for the VQ value 2 is 15. 

[0075] With reference to FIG. 11 B, an operating flow of an operation performed through the object recognizing 
processing unit 41 shown in FIG.7 will be described. As mentioned above, the object extracting unit 22 provides the 
trigger signal to the object recognizing processing unit 41 and the object-rectangle memory 9 provides the scope data 
to the same unit 41. Such operations are performed after the step S108 shown in FIG.11A has provided the determi- 
nation result YES for the currently processed image part and stores data of the outline of the image part in the object- 
rectangle memory 9. The steps S101, S102, S103, S104, S105, S106, S107, and S108 shown in FIG.11A are sub- 
stantially identical to those shown in FIG. 2. The scope data provided by the object-rectangle memory 9 consists of the 
data indicating the outline of the relevant image part. FIG. 12 shows an example of an edge of the color image processed 
in the system shown in FIG. 6. In FIG. 6, each square is the respective small region. The hatched small regions located 
lower part of FIG. 12 are small regions, among those constituting the input color image, each of the small regions being 
completely geometrically included in the outline 501 of the relevant image part. The histogram producing unit 42, in a 
step S203 shown in FIG. 1 1 B, determines small regions completely geometrically included in the outline 501 using the 
above-mentioned scope data provided by the object- rectangle memory 9. The histogram producing unit 42 thus uses 
the VQ values-determined for each small region of the small regions constituting the input color image. The VP values, 
such as those shown in FIG. 10, were stored in the vector quantized value memory 25 as mentioned above. Using the 
VQ values as mentioned above, in a step S204 shown in FIG.11B, the histogram producing unit 42 produces the 
histogram of the vector quantized values such as those described above for the thus determined small regions com- 
pletely geometrically included in the outline 501 . If the step S1 05 shown in FIG.1 1 A determines that the relevant image 
part is not one obtained as a result of the object being rotated, the object- rectangle memory 9 provides the scope data 
indicating an outline of the circumscribing rectangle which was produced for the relevant image part through the step 
S102 such as the rectangle 202 shown in FIG.3 or 301 shown in FIG.4, instead of the outline of the relevant image 
part such as the rectangle 203 shown in FIG. 3 or 302 shown in FIG.4. The histogram producing unit 42 uses the scope 
data indicating the outline of the circumscribing rectangle. Using the scope data, the histogram producing unit 42 
determines small regions, among the small regions constituting the input color image, completely geometrically includ- 
ed in the outline indicated by the scope data. The unit 42 thus produces in the step S204 the histogram of the vector 
quantized values for the thus-determined small regions. 

[0076] With reference to FIG. 13, a. step S205 shown in FIG.11B performed by the matching unit 43 will now be 
described. The step S205 compares the histogram of the vector quantized values obtained through the step S204 with 
each one of the histograms of the vector quantized values stored in the dictionary 44 shown in FIG.7. A histogram of 
vector quantized values 601 is one example of that obtained for the relevant image part through the step S204 as 
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h,Z!~ r Till ^ amP ' e ' " iS aSSUmed thatthecode °ook information 402 shown in FIG.9 has 64 chromaticity 
ZZnfl 1 3 °, ST 9 6 ? V ° Va,US iS 3 ValU6 am ° ng 64 ValU6S ' ° ^ 63 - ln the histogram 601 

vaZ t K J, nh S JT? I*' S ' 8an 3X,S ° f V6Ct0r qUantiZSd V3lUe dimensions for the above-mentioned 64 VQ 
mt VQ vaS o tSLnh T ""■ " k™ T °' appearance ^endes, indicating how many times each one of 
™in?nf th ? T PPearS m thS above - mentio ' 1ed small regions completely geometrically included in the 
th ™ T ' mage "T ° f Circumscribin 9 rectan 9le. A height of each of the bars upward projecting from 
he respective vector quantized value dimension represents the relevant appearance frequency 

S ri ^f° 9rarT ! S ° f V8Ct0r qUamiZed ValUSS T °' Tl> T * - Sh0wn in FIG ' 13 are ,h °se included in the above-men- 
Z r JZr * ' r mat '° h n S !°: Sd in d ' Cti0nary 44 Sh ° Wn in FiG - 7 - As mentioned a bove, in the object extraction 
'Tom Th SVStem !? flfth embodiment ° f Present invention, the above-mentioned object, to be extracted 

men, ? J h " ^ reC ° 9mZe ^ thus - extracted one as ^ object, includes a plurality of objects. The above- 
mentioned histograms of vector quantized values T 0 , T„ T 2 , ... corresponds to the plurality of objects. Each of these 
h tograms was prev.ously produced in a manner and has a form the same as the manner and the form in w! ch the 
histogram 601 shown in FIG.13 is produced through the step S204 

T2nJ«LTZTT ne( t- inf0rmati ° n may bS Pr ° dUCed in 3 manner simi,ar t0 one in w hich the above- 

mentioned code book .nformat.cn ,s produced. That is, the histograms of vector quantized values constitute the dic- 

n ZSJ TH- atl ° n h may , ,nClUde hiSt09ramS ° f VeCt ° r quantized values opined from not only the above-menioned 

Lr nn TnT^'T' * T Ma * es obtained as a °' Pacing the code book information by the cW 
tenng of data obtained from the small regions of many images 

TJ«lJ~lT Ch,n9 «T ? ? ° bjeCt rec °9 ni2in 9 Pressing unit 41 compares, in a step 3205. the vector quan- 
SoecSfw h t ' St ° gram h 01 ° f 5" rel6Vant ima9e Part With e3Ch 0f those constitutin 9 the dictionary information 602 
dim^l y 'h h matching unrt 43 obtains a difference, in the appearance frequencies for each vector quantized value 
dmens.on, between the histogram 601 of the relevant image part and each of those of the dictionary information 
oSS d«J 3 " 9 1 h- abOVe - mentioned diff — resulting in the above-mentioned distances DCj, SingTet us 

iS ° btained ' 33 8 reSUlt ° f thS C ° mpariSOn ° f the hist °9 ram 601 and each of the 
fo2^,Iaton (S- "" 3 mannSr Sim " ar t0 ° btaininS thS above - mentioned dista ^e DCj, by, for example, the 



DTk= 2 (VQj-Tkj) 2 (5) 
>0 



Lit r," \ ' S f™* t0 thS appearance ^iiency at the VQ value dimension 'J' of the histogram 601 . The 
the dlf f • ? I t0 6 appearance 1rs ^ e ^ at the VQ value dimension T of the histogram Tk included in 

^ h 7 ' nf0rma 6 ° 2 ' Th6n ' 3 hiSt ° gram iS SeleCted from amon 9 the histograms T 0 , T„ T 2 the thus- 

SSJv T fl,8 T T 9 3 xL St09ram haVing the minimUm distance DTk amon 9 th °se which the histograms of the 
SETSSSTT'T US ' an ° bieCt iS d6temiined ' fr ° m am ° ng thS ^-mentioned Plurality of objects (ref 
erence objects), as the object corresponding to the currently processed relevant image part. That is the currently 

SET! ? t ' S J dentified t0 C ° rreSPOnd t0 the "determined object among the plurality of ob ects ' 
oarts inc.12 X 3103 pr0V,d6s the germination result NO, a step S112 determines whether or not all the image 
parts included ,n the ,nput color image have been processed. If the determination result is NO, the step SI 03 is exe- 

maoe ** determination result N0 ' a * ap S1 12 determines whether or not alUhe 

Tit f h u V. mP C0,0r ' ma9e haVe been P roce ssed. If the determination result is NO. the step S103 

is executed. However, ,f the step 81 08 provides the determination result YES, the subsequent steps S204 and S205 
are Performed and the object extraction and recognition operation for the input color image is finished. If the step S1 1 2 

ZZtl H G t T T ati °" reSUlt YES> that iS ' " the Step 3108 determines the current 'y Processed image part not to 
correspond to the above-ment.oned object for all the image parts present in the input color image, a step si 1 3 deter- 

r^TT Paft C0rrespondin g t0 the above-mentioned object is present in the input color image. Then, the object 
extract.cn and recognition operation for the input color image is finished 

™,v tnrLTT SXtraCti0n reC ° gniti0n SySt6m in the fifth embodiment of the present invention, once the 
Zn m ? T 9e Part 13 determined t0 correspond to the object, the object extraction and recognition oper- 
l rh In 1 T C0 '° r ' m T iS finiSh6d - H0W6Ver ' the present inven «on does not limit an operation flow thereof to 

S 08 deteSr: T !t T '° ^ ^ ^ preSent in the inout ^ even if the step 

SI 08 determines a plurahty of image parts to correspond to the above-mentioned object 

[0082] Wrth reference to FIG.7, the image data processing (the appropriate steps of the steps S1 01 , S1 02, S1 03, 
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S104, S105, S1 06, S107, S108, and S1 12 shown in FIG. 11 A) through the object extracting unit 22 and object-rectangle 
memory 9 is performed in parallel with or simultaneously with the image data processing (the steps S201 , S202A, 
S202B, S203 : S204, and S205) through the vector quantizing processing unit 35, vector quantized value memory 25, 
and object recognizing processing unit 41, in the object extraction and recognition system in the fifth embodiment of 
the present invention. Further, with reference to FIG. 11 A, the circumscribing rectangle is extracted from the input color 
image, the thus-extracted rectangle then undergoes the size determination process (steps S1 03 and S1 08), the image 
part of the rectangle, which has been determined to correspond to the above-mentioned object through the size de- 
termination, then undergoes the recognition process (step S205), one by one for the image parts included in the input 
color image. Thus, it is possible to achieve real-time data processing. As a result, the object extraction and recognition 
operation can be performed accurately in high speed. Further, using the vector quantizing technique in the object 
recognition process enables effective compression of data amount being processed using full information associated 
with the image part to undergo the object recognition process. Specifically, for example, it is possible to reduce a scale 
of the dictionary 44. 

[0083] A color image recognition system in a sixth embodiment of color image recognition method and apparatus 
according to the present invention will now be described with reference to FIGS. 14 and 15. The structure of the color 
image recognition system in the sixth embodiment is similar to that of the object extraction and recognition system in 
the fifth embodiment shown in FIGS.6 and 7. However, differently from the structure of the system shown in FIGS. 6 
and 7, the image recognition system in the sixth embodiment shown in FIG. 14 does not include a function performing 
the above-described object extraction operation shown in FIG. 11 A. Elements of the system shown in FIG. 14 substan- 
tially identical to those of the system shown in FIGS.6 and 7 have the same reference numerals as those of the cor- 
responding elements of the system shown in FIGS.6 and 7, description of these element being thus omitted. 
[0084] The color image recognition system in the sixth embodiment shown in FIG.14 operates generally identically 
to the operation performed by the system shown in FIGS.6 and 7. However, differently from the operation of the system 
shown in FIGS.6 and 7, the system shown in FIG.14 performs the color image recognizing operation shown in FIG. 
15, that is, the object recognizing operation in the system show in FIGS.6 and 7, for the entirety of the input color 
image. The steps S201 , S202A, S202B, S204, and S205 are substantially the same as the steps having the same step 
numbers shown in FIG. 11 B, respectively. Further, a characteristic memory 37A is used to temporarily store the chro- 
maticities Pr and Pg therein obtained through the above-mentioned equations (1) and (2) by characteristic extracting 
unit 37. The vector quantizing executing unit 38 uses the thus-stored the chromaticities Pr and Pg as described above. 
The system in the sixth embodiment shown in FIG.14 performs substantially identically to the operation performed by 
the system in the fifth embodiment shown in FIGS.6 and 7, except for the above mentioned points. 
[0085] The image recognizing processing unit 41 shown in FIG.14, in the step S205 shown in FIG. 15, identifies the 
currently processed color image which is represented by the input color-image signal (signals R, G and B) to correspond 
to an image among a plurality of images. Each of the plurality of images is represented by a respective one of the 
histograms of vector quantized values T 0 , T 1f T 2 , ... shown in FIG.1 3. That is, the unit 41 compares the histogram 601 
representing the currently processed image with each of the histograms of vector quantized values T 0 , T^ T 2 , ... in- 
cluded in the dictionary information 602 so as to identify the currently processed image to correspond to one of the 
plurality of images (reference images). Thus, for example, in the above-mentioned example in which such image rec- 
ognition method may be applied to works in a police office, a face of a suspected person or a montage picture of the 
face may be identified to correspond to one included in the database of criminals' face photographs. 
[0086] According to the color image recognition system in the sixth embodiment, advantages, the same as those 
obtained through the applying of the vector quantizing operation in the above-described system in the fifth embodiment 
will be obtained. 

[0087] A color image recognition system in a seventh embodiment of color image recognition method and apparatus 
according to the present invention will now be described. The system in the seventh embodiment is substantially iden- 
tical to the above-described system in the sixth embodiment, and performs substantially identically to the operation 
performed by the above-described system in the sixth embodiment, except for the following point: 
[0088] In the system in the above-described sixth embodiments, the mesh dividing unit 36 divides the small regions 
as described above and obtains image data of, for example, the 64 pixels x 64 pixels for each small region to be 
processed by the characteristic extracting unit 37. However, in the system in the seventh embodiment, the number of 
pixels of which image data obtained to be processed by the unit 37 is reduced. Specifically, for example, pixels are 
sampled from the 64 pixels x 64 pixels in each small region so that 7 pixels, along each of the vertical and horizontal 
directions, are left between each two pixels to be sampled. FIG. 16 illustrates the manner of such a sampling operation. 
FIG. 1 6 shows a corner of the above-mentioned color image, similar to that shown in FIG. 8. Similarly, a finer grid shown 
in FIG. 8 indicates pixels of pixels constituting the relevant color images and a coarser grid indicates the small regions 
divided by the mesh dividing unit 36. In the example, each small region of the small regions consists of 64 pixels x 64 
pixels. With reference to FIG.1 6 in comparison to FIG.8, hatched squares among squares shown in FIG.1 6 are the 
small regions to be actually used in obtaining the chromaticities Pr and Pg. Thus, the number of pixels to be sampled 
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is a number of 1/8 x 1/8 the 64 x 64, that is, 64-1/8 x 64-1/8, for each small region. Image data of the thus-reduced 
number of pixels, that is, the thus compressed image data may be processed by the characteristic extracting unit 37 
to obtain the chromaticities Pr and Pg. By reducing the number of pixels to be used as described above, it is possible 
to effectively reduce the scale of the relevant parts of the system and effectively reduce time required forthe processinq 
of the pixel data. 

[0089] A color image recognition system in a eighth embodiment of color image recognition method and apparatus 
according to the present invention will now be described. The system in the eighth embodiment is substantially identical 
to the above-described system in the sixth embodiment, and performs substantially identically to the operation per- 
formed by the above-described system in the sixth embodiment except for the following point: 
[0090] According to the above-described sixth embodiment, the chromaticity histogram has 51 2 chromaticity dimen- 
sions as described above in the description of the fifth embodiment with reference to FIG.9. Since the number of the 
chromaticity dimensions is relatively large, the relevant memory capacity is required to be large and time required to 
compare the input image with the reference images (matching) is long. 

[0091] In the eighth embodiment, the number of the chromaticity dimensions is reduced to, for example, 64 dimen- 
sions. The chromaticity histograms obtained using the thus-reduced number of the chromaticity dimensions are then 
used in the subsequent image recognizing process (S202B, S204. and S205 shown in FIG. 15). The reduction of the 
chromaticity dimensions is performed as follows: 

[0092] First, the above-mentioned Pr and Pg are obtained through the above-mentioned equations (1 ) and (2). Then, 

20 |f Pr<64, new Pr = 0; 

if 64<Pr<1 92, new Pr = (Pr-64)/4; and 

if 192<Pr, new Pr = 31. 

If Pg<64, new Pg = 32; 

if 64<Pr<192, new Pg = (Pr-64)/4+32; and 
25 if 192<Pr, new Pg = 63. 

[0093] Then, the chromaticities new Pr 0 through 31 corresponds to the 32 chromaticity dimensions (i = 0 through 
31) and the chromaticities new Pg 32 through 63 corresponds to the 32 chromaticity dimensions (i = 32 through 63). 
Thus, the chromaticity histograms, each having 32 chromaticity dimensions are produced. 

[0094] According to the systems in the seventh and eighth embodiments, since data amount can be effectively re- 
duced. 

[0095] A color image recognition system in a ninth embodiment of color image recognition method and apparatus 
according to the present invention will now be described. The system in the ninth embodiment is substantially identical 
to the above-described system in the sixth embodiment, and performs substantially identically to the operation per- 
formed by the above-described system in the sixth embodiment, except for the following point: 
[0096] The above point relates to the comparison (matching) operation of the input image with the plurality of images 
included in the dictionary information 602 performed by the image recognizing processing unit 41 shown in FIG.14. A 
threshold value for the distances DTk, which are shown in FIG. 13, is previously provided. With reference to FIG. 17, 
in a step S206, if the distance DTk is equal to or smaller than the threshold value, it is determined that the relevant 
histogram Tk among those T v T 2 , T 3 , ... included in the dictionary information 602 may correspond to the input image 
However, if the distance DTk is larger than the threshold value, it is determined that the relevant histogram Tk among 
those T 1: T 2 , T 3 , ... included in the dictionary information 602 does not correspond to the input image. Thus, candidates 
of the histograms are determined from among those T 1( T 2 , T 3 , .... If no candidates are present in the dictionary infor- 
mation 602, it is determined in a step S209 that no reference image corresponds to (matches) the input image. If it is 
determined that some candidates are present as a result of the step S206, a step S208 determines one of the thus- 
determined candidates, which one has the minimum distance DTk among those of the candidates. By this method, it 
is possible to determine that the input image does not match any of the images included in the dictionary information. 
Thus, it is possible to improve accuracy in the image recognition. 

[0097] A color image recognition system in a tenth embodiment of color image recognition method and apparatus 
according to the present invention will now be described. The system in the ninth embodiment is substantially identical 
to the above-described system in the ninth embodiment, and performs substantially identically to the operation per- 
rormed by the above-described system in the ninth embodiment, except for the following point: 
[0098] The point relates to the above-mentioned threshold value for the distances DTk provided in the above-de- 
scribed ninth embodiment. In the system in the tenth embodiment, the above-mentioned threshold value is determined 
depending on each image among the images which are stored as the histograms T,. T 2 , T 3 , ... in the dictionary 44 
Thus, it is possible to provide a number of threshold values different from one another, the number being the same as 
the number of histograms T 2; T 3 , .... In the histogram comparison step S206 shown in FIG.17, the distance DTk 
(DT 1f DT 2> DT 3 , ...) is obtained for each one of the histograms T, , T 2 , T 3 , .... Then, each of the thus-obtained distances 
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DTk is compared with the respective one of the threshold values given to the histograms T 1s T 2f T 3 , .... Thus, the 
candidate is determined such that the histogram, the distance DTk of which is equal to or smaller than the relevant 
threshold value ; can be the candidate. If the distance DTk is larger than the relevant threshold value, the relevant 
histogram cannot be the candidate. By this method, it is possible to perform the image matching processing flexibly to 

5 be appropriate to characteristics of each image of the images, the histograms of which are stored in the dictionary 44. 
Specifically, for example, an image k among the images stored in the dictionary in the form of the histograms has 
characteristics such that the image k is very similar in their histograms to another image j. In such a case, the threshold 
value for the image k may be small enough so that, if the input image actually corresponds to the image j, it is possible 
to determine the input image does not match the image k in the step S206. Thus, it is possible to prevent such a similar 

io but different image from being determined to match the relevant image k in the step S208. Thus, it is possible to further 
improve accuracy in the image recognition. 

[0099] A color image recognition system in a eleventh embodiment of color image recognition method and apparatus 
according to the present invention will now be described with reference to FIG.1 8. The system in the eleventh embod- 
iment is substantially identical to the above-described system in the sixth embodiment shown in FIG. 14, and performs 
is substantially identically to the operation performed by the above-described system in the sixth embodiment, except 
for certain points which will be described later. Elements of the system shown in FIG.1 8 substantially identical to those 
of the system shown in FIG. 14 have the same reference numerals as those of the corresponding elements of the 
system shown in FIG 14, description of these element being thus omitted. 

[0100] The control unit 27 A overall controls the other units shown in FIG. 18. The characteristic extracting unit 37B 
20 and a conversion compressing table 50 produce chromaticity histograms different from the chromaticity histograms 
produced by the characteristic extracting unit 37 shown in FIG. 14. Similarly, the code book stores the code book in- 
formation consisting of the characteristic histograms previously provided in a manner the same as the manner in which 
the chromaticity histogram extracting unit 37B and conversion compressing table 50 produce the chromaticity histo- 
grams. The chromaticity histograms are produced for each small region of the above-described small regions consti- 
25 tuting the input color image similarly to the relevant operation in the system in the sixth embodiment. The thus-produced 
chromaticity histograms are stored in the characteristic memory 37C and then used by the vector quantizing unit 38 
similarly to the relevant operation in the system in the sixth embodiment. 

[01 01 ] The characteristic extracting unit 37B, for each pixel of the pixels constituting the relevant small region, uses 
the input color image signals R, G and B and thus obtains the above-mentioned chromaticities Pr, Pg and Pb through, 
30 for example, the following equations (1), (2) and (3): 

Pr=256.R/(R+G+B) (1), 



35 Pg=256.G/(R+G+B) (2), . 

and 

40 

Pb=256.B/(R+G+B) (3). 

The characteristic extracting unit 37B causes the thus-obtained chromaticities Pr, Pg and Pb to undergo data com- 
pression process using the conversion compressing table 50. One embodiment of the data compression process will 

45 now be described with reference to FIG.1 9. According to the compression converting table 50 shown in FIG.1 9, for 
each chromaticity of the chromaticities Pr, Pg and Pb, a chromaticity range consisting of the 256 chromaticities 0 
through 255 is converted into a chromaticity range consisting of 1 6 chromaticities 0 through 15. For example, a chro- 
maticity 255 is converted into 15. Further, the data conversion of the conversion compressing table 50 is performed 
according to a conversion curve 51, not linear, shown in FIG.19. That is, in a part where the input value is a small 

so value, that is, the left side in FIG.19, a a small number of the input values are converted into the single output value. 
However, in a part where the input value is a large value, that is, the left side in FIG.19, a large number of the input 
values are converted into the single output value. 

[01 02] The characteristic extracting unit 37B uses the thus-obtained chromaticities Pr, Pg and Pb, and produces the 
chromaticity histograms 401 A. an example of which is shown in FIG.19. The above-mentioned conversion using the 
55 conversion compressing table 50 can reduce the chromaticity dimensions which corresponds to the scale of each of 
the chromaticity histograms. Thus, the relevant memory capacity can be reduced and also the matching processing 
(image recognition operation) can be performed in high speed. 

[01 03] As shown in FIG. 1 9 : each of the histogram 401 A has the horizontal axis of the 47 chromaticity dimensions (i 
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a 0 through 48). Among the 47 chromaticity dimensions, the first 16 dimensions (i = 0 through 15) correspond to the 
16 chromaticities of Pr, the second 16 dimensions (i = 16 through 31) correspond to the 1 6 chromaticities of Pg, the 
last 1 6 dimensions (i = 32 through 47) correspond to the 1 6 chromaticities of Pb. Similar to the histograms 401 shown 
in FIG. 9, a height of each of the bars upward projecting from the respective chromaticity dimension represents the 
relevant appearance frequency of the chromaticities for the relevant small region. 

[0104] The data compression ratio is not limited to the above-mentioned ratio of 1 6 to 256. Various data compression 
ratios can be applied appropriately. Further, the above-described characteristic extracting operation is not limited to 
that using the chromaticities Pr, Pg and Pb. Instead of the chromaticities, the YMC (Yellow, Magenta and Cyan) signals 
may be used as characteristic values to produce histograms instead of the chromaticity histograms. The YMC signals 
are obtained as a result of appropriately converting the RGB signals. Further, the Lab signals (corresponding to the 
values in the known L*a*b* color space, that is, CIE1976 L*a*b* color space) may be used as characteristic values to 
produce histograms instead of the chromaticity histograms. 

[0105] FIG.20 is identical to FIG. 9, except for the number of chromaticity dimensions of each chromaticity histogram 
of the histogram 401 or 401 A of the input image and the histograms C 0 , C 1( C 2 , ... of the code book information 402 
or 402A. In FIG.9, the number of chromaticity dimensions is 512 while in FIG.20, the same is 48. 
[0106] A color image recognition system in a twelfth embodiment of color image recognition method and apparatus 
according to the present invention will now be described. The system in the twelfth embodiment is substantially identical 
to the above-described system in the eleventh embodiment, and performs substantially identically to the operation 
performed by the above-described system in the eleventh embodiment, except for the following point: 
[0107] The operation performed by the vector quantizing unit 38 is different from that in the system in the eleventh 
embodiment. In the system in the eleventh embodiment, the chromaticity histograms produced for all the small regions 
are uniformly compared with all the histograms stored in the code book 39A. As a result the VQ values are given to 
the small regions as shown in FIG. 10. However, in the system in the twelfth embodiment, manners in which the histo- 
grams of the small regions are compared with the code book information are different among the small regions. The 
manners are determined as a result of examining the relevant histograms of the small regions. Thus, the small regions 
constituting certain part in the input image are substantially omitted from being used to compared with the code book 
information. The above-mentioned certain part is, for example, in the input image, a background with respect of a 
relevant foreground, or a white sheet being exposed around a something printed on the white sheet, each of the 
foreground and the something being essential to be recognized but each of the background and the exposed white 
sheet being not necessary to be examined. Each of the background and the white sheet may include therein noise 
patterns which may adversely affect the proper recognition performance of the color image recognition system. By 
removing such extra data from being substantially examined, it is possible to improve recognition (identification) suc- 
cess ratio and to improve data processing rate (percentage). 

[0108] Generally speaking, in such a background or an exposed white sheet, image density is uniform in each of the 
small regions. As a result, the chromaticity histogram produced for the small region should be such that the appearance 
frequencies at specific concentrated ones among the series of chromaticity dimensions in the horizontal axis scale are 
very large, and those at the other ones are very small. That is, it is likely that the maximum value of the appearance 
frequencies is large, and those of the significant values are distributed at a narrow extent in the horizontal axis scale. 
Such a general phenomenon is used so that if the maximum value of the appearance frequencies in the relevant small 
region exceeds a predetermined threshold value, the vector quantizing unit 38 determines that the relevant small region 
corresponds to an image part in which an image density is uniform all over, that is, the above-mentioned certain part 
which should be removed from further examination. The unit 38 thus gives a specific VQ value, such as "0" to the 
relevant small region. The unit 38 then treats the small regions having the VQ value "0" as ones to be ignored. Instead 
of giving the specific VQ value if the maximum appearance frequency exceeds the predetermined value, it is also 
possible to prevent the comparison with the code book information 402A if an appearance frequency exceeds a pre- 
determined value. Thus, a certain part in the input image can be removed from being further processed (from undergoing 
the vector quantizing process) , the certain part being a part in which color is generally uniform all over. 
[01 09] Instead of determining such a part in the input image as that to be omitted from being further processed using 
the heights of the histograms as described above, it is also possibleto further use information of awidth in the histogram, 
for which width appearance frequencies of significant values are concentratedly distributed, for the same determination! 
[0110] Further for the same determination, it is also possible to use information as to which part in the horizontal 
axis scaie the appearance frequencies of significant values are distributed. For example, a white background part is 
found out to be omitted from being further processed. By such a method, it is possible to remove noise image information 
from being further processed. As described above, the noise image information maybe included in relevant information 
as a result of, for example, an image scanner reads in information of not only an relevant image printed on a relevant 
sheet but also another image which is printed on another sheet subsequent to the relevant sheet. In such a case, the 
white background part even including such noise are generally white so that the above-mentioned method in which 
information as to which part in the horizontal axis scale the appearance frequencies of significant values are distributed 
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is used. 

[0111] Further instead of determining such a part in the input image as that to be omitted from being further processed 
using the heights of, widths of, and positions in the histograms as described above, other characteristics may be used 
for the same purpose. The other characteristics are those obtained as a result of analyzing the histograms. 

5 [01 12] A color image recognition system in a thirteenth embodiment of color image recognition method and apparatus 
according to the present invention will now be described. The system in the thirteenth embodiment is substantially 
identical to the above-described system in the eleventh embodiment, and performs substantially identically to the op- 
eration performed by the above-described system in the eleventh embodiment., except for the following point: 
[01 13] In the system in the eleventh embodiment, a VQ value are given to each small region if the above-described 

10 distance DCj between the histogram 401 A of the small region and that of the relevant histogram of the code book 
information 402A is the smallest among the histograms of the code book information 402A. However as described 
above, the code book information may be formed as a result of clustering a huge number of various images. As a 
result, if the input image is different from any one of the images, the histograms of which are stored as the dictionary 
information 602 shown in FIG. 1 3, the distance DCj between the histogram of the input image and each of the histograms 

*s of the code book information may be large. In this embodiment, if the minimum distance DCj has a value larger than 
a predetermined threshold value, no VQ value is given to the relevant small region in the vector quantizing unit 38. As 
a result, the matching (comparison with the dictionary information) process performed by the image recognizing unit 
41 using the thus produced VQ values can be simplified in such a case. As a result, it is possible that the image 
recognizing unit 41 determines that the dictionary 44 has no data matching the input image if no significant VQ values 

20 are given to the input image. Thus, it is possible to achieve high-speed determination. 

[0114] A color image recognition system in a fourteenth embodiment of color image recognition method and appa- 
ratus according to the present invention will now be described. The system in the fourteenth embodiment is substantially 
identical to the above-described system in the eleventh embodiment, and performs substantially identically to the op- 
eration performed by the above-described system in the eleventh embodiment, except for the following point: 

25 [01 1 5] The above point relates to the comparison (matching) operation of the input image with the plurality of images 
included in the dictionary information 602 performed by the image recognizing processing unit 41 shown in FIG.1 8. A 
threshold value for the distances DTk, which are shown in FIG.13, is previously provided. With reference to FIG. 17, 
in a step S206, if the distance DTk is equal to or smaller than the threshold value, it is determined that the relevant 
histogram Tk among those T 1s T 2 , T 3 , ... included in the dictionary information 602 may correspond to the input image. 

30 However, if the distance DTk is larger than the threshold value, it is determined that the relevant histogram Tk among 
those T n , T 2 ,T 3 , ... included in the dictionary information 602 does not correspond to the input image. Thus : candidates 
of the histograms are determined from among those T 1f T 2 , T 3: .... If no candidates are present in the dictionary infor- 
mation 602, it is determined in a step S209 that no reference image corresponds to (matches) the input image. If it is 
determined that some candidates are present as a result of the step S206, a step S208 determines one of the thus- 

35 determined candidates, which one has the minimum distance DTk among those of the candidates. By this method, it 
is possible to determine that the input image does not match any of the images included in the dictionary information. 
Thus, it is possible to improve accuracy in the image recognition. 

[01 16] A color image recognition system in a fifteenth embodiment of color image recognition method and apparatus 
according to the present invention will now be described. The system in the fifteenth embodiment is substantially 

40 identical to the above-described system in the fourteenth embodiment, and performs substantially identically to the 
operation performed by the above-described system in the fourteenth embodiment, except for the following point: 
[0117] The point relates to the above-mentioned threshold value for the distances DTk provided in the above-de- 
scribed fourteenth embodiment. In the system in the fifteenth embodiment, the above-mentioned threshold value is 
determined depending on each image among the images which are stored as the histograms T^ T 2 , T 3 , ... in the 

^5 dictionary 44. Thus, it is possible to provide a number of threshold values different from one another, the number being 
the same as the number of histograms T 1f T 2 , T 3 , .... In the histogram comparison step S206 shown in FIG. 17, the 
distance DTk (DT V DT 2 , DT 3 , ...) is obtained for each one of the histograms T 1( T 2 , T 3 , .... Then, each of the thus- 
obtained distances DTk is compared with the respective one of the threshold values given to the histograms T v T 2 , 
T 3 , .... Thus ; the candidate is determined such that the histogram, the distance DTk of which is equal to or smaller than 

50 the relevant threshold value, can be the candidate. If the distance DTk is larger than the relevant threshold value, the 
relevant histogram cannot be the candidate. Sy this method, it is possible to perform the image matching processing 
flexibly to be appropriate to characteristics of each image of the images, the histograms of which are stored in the 
dictionary 44. Specifically, for example, an image k among the images stored in the dictionary in the form of the histo- 
grams has characteristics such that the image k is very similar in their histograms to another image j. In such a case, 

55 the threshold value for the image k may be small enough so that, if the input image actually corresponds to the image 
j, it is possible to determine the input image does not match the image k in the step S206. Thus, it is possible to prevent 
such a similar but different image from being determined to match the relevant image k in the step S208. Thus, it is 
possible to further improve accuracy in the image recognition. 
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Son's mavtr m t a h d? r TVr n !i° n 1 * ^ above " desc *ed embodiments, and variations and modifi- 

cations may be made wrthout departing from the scope of the present invention as defined by the appended claims. 

Claims 

1 . A color image recognition method, characterized in that: 

said method comprises steps of: 

a) (S202A, S202B) extracting characteristics from an input color image signal - 

b) (S101 through S108) finding a candidate image part corresponding to a plurality of possible objects 

milt P Tl ' mage represented b V said ^ image signal, in paral.el with said step a) by 
means of an object extraction method; and H ' y 

c) (S205) selecting one of said plurality of possible objects as relevant by using said extracted character- 
istics which correspond to the candidate image part, acieacnaracter 

2. The color image recognition method according to claim 1 , characterized in that: 

said step c) is executed only after a time said step b) had found said candidate image part. 

3. The color image recognition method according to claim 1 or 2 wherein said step b) comprises the steps of: 

ba) (S102) deteimining a circumscribing rectangle from an input two-tone image signal- 

SL C 'S^ in H rSCt f T beinS 3 rSCtan9le circumscribin 9 an image part which consists of continuous 
pixels, each of said pixels being a pixel of a predetermined same tone- 

| b mage 1 part; and**" 9 C ° ntaCt P ° Siti ° nS * * M circumscribin 9 "^ng.e comes into contact with said 

2 !fJ° 6, S1 ? 7 ' S J° 8 ' S115 ' 311 6) detemini "3 whether or not said image part is identical to a given object 
the determination being performed using said contact positions. 

4. The color image recognition method of claim 3, characterized in that: 

feng^ 

said step be) comprises steps of: 

tai?i°?w S1 °? de f rminin9 ' based on an ^sumption that said image part has a rectangular shape, 
lengths of sides of said rectangular shape of said image part; 

bc-2) (S108) comparing said lengths of sides of said rectangular shape of said image part with said ore- 
determined lengths of sides of said rectangular shape of said objects and 

bc-3) (S115, S116) determining whether or not said image part is identical to said given object, based on 
a result of comparison performed through said step bc-2). 

5. The color image recognition method of claim 4, characterized in that: 

said step b) further comprises a step of 

^^ISST^S^ not said ima9e part is an ima9e part obtained as a resuit of ™^ said 

and wherein: 

said step bc-1) can properly determine lengths of sides of said rectangular shape of 
said image part using a determination result given by said step bd). 

6. The color image recognition method according to claim 3, characterized in that: 

said step b) further comprises steps of: 
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be-1 ) (S1 01 ) obtaining a first one of said input two-tone image signal from an input color image signal, in 
a first manner, as a result of comparing a lightness value represented by said input color image signal 
with a first threshold value; 

be-2) (S1 01 ) obtaining a second one of said inputtwo-tone image signal from said input color image signal, 
5 in a second manner, as a result of comparing said lightness value represented by said input color image 

signal with a second threshold value; 

and wherein said step ba) determines a first one of said circumscribing rectangle from said first one of 
said input two-tone image signal, and determines a second one of said circumscribing rectangle from said 
second one of said input two-tone image signal. 

10 

7. The object extraction method according to claim 6, characterized in that: 
said step b) further comprises steps of 

bf) determining whether or not said first one of said circumscribing rectangle geometrically completely includes 
said second one of said circumscribing rectangle, and determining whether or not said second one of said 
circumscribing rectangle geometrically completely includes said first one of said circumscribing rectangle; and 
be) removing a certain one of said first one and second one of said circumscribing rectangles from undergoing 
said steps bb) and be), if said step bf) has determined that said certain one geometrically completely includes 
a remaining one of said first one and second one of said circumscribing rectangles. 

8. A color image recognition method according to claim 1 , characterized in that: 

a step a) comprises the substeps of 
a1) (S201) dividing an input color image into small regions; 

a2) (S202A) extracting characteristics from said smail regions of said input color image signal; 

a step c) comprises the substeps of: 

c1 ) (S204) selecting a reference characteristic collection from among a plurality of reference characteristic 
collections for each of said small regions, such that the thus-selected reference characteristic collection has 
characteristics having the highest degree of similarity to said characteristics extracted from said each of said 
small regions; 

each of said plurality of reference characteristic collections having an identification number and thus the iden- 
tification number of the thus-selected reference characteristic collection being given to said each of said small 
regions; 
and 

c2) (S205) selecting an object from among said plurality of possible objects as relevant such that: 
the thus-selected object is previously divided into the small regions, 

the identification number of a reference characteristic collection among said plurality of reference char- 
acteristic collections is previously given to each of said small regions of said thus-selected object, and 
characteristics of a set of the identification numbers given to said small regions of said thus-selected object 
have the highest degree of similarity to characteristics of a set of the identification numbers corresponding 
to the thus-selected object. 

color image recognition method according to claim 8, characterized in that: 

said step a2) comprises steps of: 

a2-1) sampling pixels from among pixels constituting each of said small regions which constitutes said 
input color image, and 

b-2) using image data of the thus-sampled pixels to extract said characteristics from said small regions. 

10. The color image recognition method according to claim 8, characterized in that: 
said method further comprises a step of 

e) converting said characteristics, which have been extracted by said step a2), so that a date amount of said 
characteristics is reduced. 
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11. The color image recognition method according to claim 8, characterized in that: 
said method further comprises steps of: 

f) obtaining a value for each of said plurality of objects, said value representing how the reference characteristic 
co lections given to said small regions of said each of said plurality of objects is similar to the reference char- 
acteristic collections given to said small regions of said input color image- and 

g) comparing said value obtained through said step f) with a threshold value so as to determine whether or 
not said each of said plurality of objects is a candidate of objects among said plurality of objects, said candidate 
of objects being those each of which have a possibility of being determined to be identical to said input color 
image. r 

12. The color image recognition method according to claim 11, characterized in that- 
said threshold value used in said step g) is determined individually for each of said plurality of objects based on 
features of said plurality of objects. 

13. The color image recognition method according to claim 8, wherein a manner of said step d) varies among said 
small regions constituting said input color image, depending on the characteristics extracted through said step a2) 
for each of said small regions constituting said input color image. 

14. The color image recognition method according to claim 8, characterized in that- 

each of the characteristic collections comprises a histogram of appearance frequencies 

the identification numbers of characteristic collections among said characteristic collections being given to the 

small regions of said input color image and said plurality of objects, 

each of said appearance frequencies being an appearance frequency of pixels having same characteristics. 

15. The color image recognition method according to claim 14, characterized in that: 

said characteristics extracted through said step a2) comprise chromaticities. 

16. The color image recognition method according to claim 14, characterized in that: 
said method further comprises a step of 

h) preventing said step d ) from being executed if the appearance frequency of the histogram of a small region 
among said small regions constituting said input color image exceeds a predetermined value. 

17. The color image recognition method according to claim 16, characterized in that: 
said method further comprises a step of 

i) giving a predetermined number to said small region, said appearance frequency of which exceeds said 
predetermined value in said step 

18. The color image recognition method according to claim 14, characterized in that: 
said method further comprises a step of 

j) preventing said step d ) from being executed if only a number of appearance frequencies among the ap- 
pearance frequencies constituting the histogram of a small region among said small regions constituting said 
input color image have significant values, 

said number of appearance frequencies being those of the characteristics approximating one another, and said 
number being less than a threshold value. 

19. The color image recognition method according to claim 18, characterized in that: 
said method further comprises a step of 

k) giving a predetermined number to said small region, said step j) having prevented said step d) from being 
executed for said small region. ' 9 
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20. The color image recognition method according to claim 8, characterized in that: 
said method further comprises a step of: 

I) preventing said step cl) from being executed 

if the characteristics of each reference characteristic collection among the plurality of reference characteristic 
collections has similarity to said characteristics extracted from said each of said small regions, said similarity 
being one of less than a predetermined degree; and 

m) giving a predetermined number to said small region, said step 1 ) having prevented said step c1 ) from being 
executed for said small region. 

21 . The color image recognition method according to claim 8, characterized in that: 
said step c2) selects the object from among said plurality of objects such that: 

the thus-selected object being previously divided into the small regions; 

the identification number of a reference characteristic collection among said plurality of reference characteristic 
collections being previously given to each of said small regions of said thus-selected object, 
a histogram of a set of the identification numbers given to said small regions of said thus-selected object 
having the highest degree of similarity to a histogram of a set of the identification numbers corresponding to 
the thus-selected object; 

each of the histograms comprising a histogram of appearance frequencies, each of said appearance frequen- 
cies being an appearance frequency of the small regions having the same identification number. 

22. A color image recognition apparatus, characterized in that: 
said apparatus comprises: 

means (37) for extracting characteristics from an input color image signal; 

means (22) for finding a candidate image part corresponding to a plurality of possible objects from an input 
color image represented by said input color image signal, in parallel with said extracting means by means of 
an object extracting means; and 

means (41) for selecting one of said plurality of possible objects as relevant by using said extracted charac- 
teristics which correspond to the candidate image part. 

23. The color image recognition apparatus according to claim 22 characterized in that: 
said means (22) for finding a candidate image part comprises: 

means (2) for determining a circumscribing rectangle from an input two-tone image signal; 

said circumscribing rectangle being a rectangle circumscribing an image part which consists of continuous 

pixels, each of said pixels being a pixel of a predetermined same tone; 

means (2) for detecting contact positions at which said circumscribing rectangle comes into contact with said 
image part; and 

means (4) for determining whether or not said image part is identical to a given object, the determination being 
performed using said contact positions. 

24. A color image recognition apparatus according to claim 22, characterized in that: 
said means for extracting comprises: 

means (36) for dividing an input color image into small regions; 
means (37) for extracting characteristics from said small regions; 
said means for selecting comprises: 

means (38) for selecting a reference characteristic collection from among a plurality of reference charac- 
teristic collections for each of said small regions, such that the thus-selected reference characteristic col- 
lection has characteristics having the highest degree of similarity to said characteristics extracted from 
said each of said small regions; 

each of said plurality of reference characteristic collections having an identification number and thus the 
identification number of the thus-selected reference characteristic collection being given to said each of 
said small regions; 
and 
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means (41 ) for selecting an object from among said plurality of objects as relevant such that: 
the thus-selected object being previously divided into the small regions 

me identification number of a reference characteristic collection among said plurality of reference 
charactensfc collect.ons be,ng previously given to each of said small regions of said thus-selected 

Sl a t? riStiCS t K ' I*? ° f ^ identification ™ mbere to said small regions of said thus-selected 
object having the highest degree of similarity to characteristics of a set of the identification numbers 
corresponding to the thus-selected object. numoera 

Patentansprilche 

1 . Farbbilderkennungsverfahren, dadurch gekennzeichnet, dass 

das Verfahren folgende Schritte umfasst: 

5 Charakteristiken werden einem Eingangs-Farbbildsignal extrahiert; 

In il I ] e c Kandidatenbildteil ' der «™ Vielzahl moglicher Objekte entspricht. wird ausgehend 
a iLT IS \ « ^ Ein 9 an 9-^bbildsignal dargesteHt wird. und zwar 

parallel zu dem Schntt a) mittels eines Objekt-Extraktionsverfahrens- und 

c) (S205) eines der Vielzahl von moglichen Objekten wird als relevant ausgewahlt, indem die extrahierten 
Charaktenstiken verwendet werden, die dem Kandidatenbildteil entsprechen extrah.erten 

2. Farbbilderkennungsverfahren nach Anspruch 1, dadurch gekennzeichnet, dass der Schritt c) erst nach einer 
Zeit ausgefuhrt wird, wenn der Schritt b) den KandidatenBildteil gefunden hat. 

3. Farbbiiderkennungsverfahren nach Anspruch 1 oder2, bei welchem der Schritt b) die folgenden Schritte umfasst: 

ba) (S102) ein umschreibendes Rechteck wird von einem Eingangs-Zweiton-Bildsignal bestimmt 

das umschreibende Rechteck ist ein Rechteck, das ein Bildteil umschreibt, das aus kontinuieriichen Pixeln 

besteht, wooe, jedes der Pixel ein Pixel eines vorbestimmten gleichen Tons isf 

BSdtSu^mmTrd" 08 '" 0 " 60 Werd6n d6tektiert ' d6nen ^ umschreibe nde Rechteck in Kontakt mit dem 
be) (S106, S107, S108, S115, S116) es wird bestimmt, ob der Bildteil mit einem gegebenen Obiekt identisch 
-st Oder n,cht, wobei die Bestimmung durchgefuhrt wird, indem die Kontaktpositionen verwendei weS 

4. Farbbilderkennungsverfahren nach Anspruch 3, dadurch gekennzeichnet, dass das gegebene Objekteine recht- 

Seis'Td ■ wobei die rechteckige Form des gegebenen objekts vorbestimmte °TsZ::i 

der Schritt be) folgende Schritte umfasst: 

JL 1 lSln 6 H S1 ° 7 L baS ^ ren t aUf 6in6r Annahm6 ' daS dSr BHdteil eine ^Meckige Form hat. werden Langen 
von Seiten der rechteckigen Form des Bildteils bestimmf 

TJTTl die . Langan der Seiten der ^chteckigen Form des Bildteiles werden mit den vorbestimmten Lan- 
gen der Seiten der rechteckigen Form der Objekte verglichen und 

bc-3) (S11 5, S1 1 6) es wird bestimmt, ob Oder ob nicht der Bildteil mit dem gegebenen Objekt identisch ist und 
zwar basierend auf einem Ergebnis des Vergleichs, der durch den Schritt tab) durchgefuhrt wfrd 

" T^T*Zr SMen n3Ch AnSPrUCh 4 ' dadUrCh *' kM <*"«*' *■« Schritt b, welter den Schritt 

bd) (S1 05) bestimmt wird, ob oder ob nicht der Bildteil ein Bildteil ist, der als ein Ergebnis des Drehens der recht 
eckigen Form des Objekts erhalten worden ist; urenens aer recht- 

und wobei: 

ll^nnty I 66 ' 9 " 61 L3n f n V ° n Seit6n dSr rechtecki 9 en F °™ des Bildteils bestimmen kann, wobei ein 
Bestimmungsergebnis verwendet wird, das durch den Schritt bd) gegeben ist. 

Schritte "mS n9SVerfahren d8dUrCh ^"^^ — = Schritt b) weiter foigende 
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be-1) (S101) ein erstes des Eingangs-Zweiton-Bildsignals von einem Eingangs-Farbbildsignal wird in einer 
ersten Art und Weise als ein Ergebnis des Vergleichens eines Helligkeitswertes : der durch das Eingangs- 
Bildsignal dargestellt wird, mit einem ersten Schwellenwert erzielt; 

be-2) (S1 01 ) ein zweites des Eingangs-Zweiton-Bildsignal wird von dem Eingang-Farbbildsignal in einer zwei- 
ten Art und Weise als ein Ergebnis des Vergleichens des Heiligkeitswertes, der durch das Eingabe-Farbbild- 
signal dargestellt wird, mit einem zweiten Schwellenwert, erzielt; 

und wobei der Schritt ba) ein erstes des umschreibenden Rechtecks von dem ersten des Eingangs-Zweiton- 
Bildsignals bestimmt und ein zweites von dem umschreibenden Rechteck von dem zweiten des Eingangs- 
Zweiton-Bildsignals bestimmt. 

7. Objekt Extraktionsverfahren nach Anspruch 6, dadurch gekennzeichnet, dass: der Schritt b) weiter die Schritte 
umfasst, wonach bf) bestimmt wird, ob oder ob nicht das erste des umschreibenden Rechtecks geometrisch voll- 
standig das zweite des umschreibenden Rechtecks enthalt und bestimmt wird, ob oder ob nicht das zweite des 
umschreibenden Rechtecks geometrisch vollstandig das erste des umschreibenden Rechtecks enthalt; und 

be) ein gewisses des ersten und zweiten umschreibenden Rechtecks wird ausgehend von einem Durchmachen 
der Schritte bb) und be) entfernt, falls der Schritt bf) bestimmt hat, dass das gewisse geometrisch vollstandig ein 
verbliebenes des ersten und zweiten der umschreibenden Rechtecke enthalt. 

8. Farbbilderkennungsverfahren nach Anspruch 1 , dadurch gekennzeichnet, dass: ein Schritt a) die Unterschritte 
umfasst, wonach 

a1) (S201) ein Eingangs-Farbbild wird in kleine Bereiche unterteilt; 

a2) (S202A) Charakteristiken werden von den kleinen Bereichen des Eingangs-Farbbildsignals extrahiert; 

ein Schritt c) umfasst die Unterschritte, wonach: 

ci) (S204) eine Referenz-Charakteristiksammlung aus einer Vielzahl von Referenz-Charakteristiksammlun- 
gen fur jede der kleinen Bereiche ausgewahlt wird, so dass die somit ausgewahlte Referenz-Charakteristik- 
sammlung Charakteristiken aufweist die den hochsten Grad an Ahnlichkeit mit den Charakteristiken haben, 
die von jedem der kleinen Bereiche extrahiert werden; 

wobei jede der Vielzahl von Referenz-Charakteristiksammlungen eine Identifikationsnummer aufweist und 
somit die Identif ikationsnummer der somit ausgewahlten Referenz-Charakteristiksammlungzu jedem der klei- 
nen Bereiche gegeben wird; und 

c2) (S205) ein Objekt wird unter der Vielzahl von moglichen Objekten als relevant ausgewahlt, so dass: 
das somit ausgewahlte Objekt vorab in kleine Bereiche unterteilt wird, 

die Identifikationsnummer einer Referenz-Charakteristiksammlung unter der Vielzahl von Referenz-Cha- 
rakteristiksammlungen wird vorab zu jedem der kleinen Bereiche des somit ausgewahlten Objekts gege- 
ben und 

Charakteristiken eines Satzes der Identifikationsnummer, die zu den kleinen Bereichen der somit ausge- 
wahlten Objekte gegeben wurden, haben den hochsten Grad an Ahnlichkeit zu Charakteristiken eines 
Satzes der Identifikationsnummern, die dem somit ausgewahlten Objekt entsprechen. 

9. Farbbilderkennungsverfahren nach Anspruch 8, dadurch gekennzeichnet, dass der Schritt a2) folgende Schritte 
umfasst: 

a2-1 ) Pixel werden ausgehend von Pixel abgetastet, die jeden der kleinen Bereiche bilden, die das Eingangs- 
Farbbild bilden, und 

b2) Bilddaten der somit abgetasteten Pixel werden verwendet, urn die Charakteristiken von den kleinen Be- 
reichen zu extrahieren. 

10. Farbbilderkennungsverfahren nach Anspruch 8, dadurch gekennzeichnet, dass das Verfahren weiter einen 
Schritt umfasst wonach 

a) die Charakteristiken, die durch den Schritt a2) extrahiert worden sind, konvertiert werden, so dass eine 
Datenmenge der Charakteristiken reduziert wird. 

1 1 . Farbbilderkennungsverfahren nach Anspruch 8, dadurch gekennzeichnet, dass: das Verfahren weiter die Schrit- 
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te umfasst, wonach: 

f) ein Wert fur jedes der Vielzahl von Objekten erhalten wird : wobei der Wert darstellt, wie die Referenz-Cha- 
rakteristiksammlungen, die fur die kieinen Bereiche eines jeden der Vielzahl von Objekten gegeben sind, den 

s Referenz-Charakteristiksammiungen ahnlich sind, die fur die kieinen Bereiche des Eingangs-Farbbiids gege- 

ben sind; und 

g) der Wert, derdurch den Schrittf) erhalten wird, wird mit einem Schwellenwert verglichen, um so zu bestim- 
men , ob oder ob nicht jedes der Vielzahl von Objekten ein Kandidat der Objekte unter der Vielzahl von Objekten 
ist, wobei der Kandidat der Objekte jener ist von denen jeder eine Moglichkeit hat, als identisch mit dem 

10 Eingangs-Farbbild bestimmt zu werden. 

12. Farbbilderkennungsverfahren nach Anspruch 11, dadurch gekennzeichnet, dass: der Schwellenwert, der im 
Schritt g) verwendet worden ist, individuell fur jeden der Vielzahl von Objekten basierend auf Merkmalen der Viel- 
zahl von Objekten bestimmt wird. 

15 

13. Farbbilderkennungsverfahren nach Anspruch 8, bei welchem eine Art und Weise des Schrittes d) unter den kiei- 
nen Bereichen variiert, die das Eingangsfarbbild bilden, und zwar in Abhangigkeit von den Charakteristiken, die 
durch den Schritt a2) fur jeden der kieinen Bereiche extrahiert werden, die das Eingangsfarbbild bilden. 

Farbbilderkennungsverfahren nach Anspruch 8, dadurch gekennzeichnet, dass: eine jede der charakteristischen 
Sammlungen ein Histogramm von Erscheinungshaufigkeiten umfasst, wobei die Identifikationsnummern der cha- 
rakteristischen Sammlungen unter den charakteristischen Sammlungen zu kieinen Bereichen des Eingangsfarb- 
bildes und der Vielzahl von Objekten gegeben werden, wobei jede der Erscheinungsfrequenzen eine Erschei- 
nungsfrequenz von Pixeln ist, die dieselben Charakteristiken aufweisen. 

Farbbilderkennungsverfahren nach Anspruch 14, dadurch gekennzeichnet, dass: die Charakteristiken, diedurch 
den Schritt a2) extrahiert werden, Chromazitaten bzw. Farbart und Farbsattigung umfassen. 

16. Farbbilderkennungsverfahren nach Anspruch 14, dadurch gekennzeichnet, dass: das Verfahren weiter den 
30 Schritt umfasst, wonach h) der Schritt c1 ) daran gehindert wird, ausgefuhrt zu werden, falls die Erscheinungshau- 

figkeit des Histogramms eines kieinen Bereichs unter den kieinen Bereichen, die das Eingangs-Farbbild bilden, 
einen vorbestimmten Wert uberschreitet. 

17. Farbbilderkennungsverfahren nach Anspruch 16, dadurch gekennzeichnet, dass das Verfahren weiter den 
35 Schritt umfasst, wonach i) eine vorbestimmte Nummer bzw. Zahl dem kieinen Bereich gegeben bzw. zugewiesen 

wird, wobei dessen Erscheinungshaufigkeit den vorbestimmten Wert im Schritt h) uberschreitet. 

18. Farbbilderkennungsverfahren nach Anspruch 14, dadurch gekennzeichnet, dass: 
40 das Verfahren weiter einen Schritt umfasst, wonach . 

j) der Schritt d) an einer Ausfuhrung gehindert wird, falls nur eine Zahl von Erscheinungshaufigkeiten 
unter den Erscheinungshaufigkeiten, die das Histogramm eines kieinen Bereichs unter den kieinen Be- 
reichen bilden, die das Eingangs-Farbbild bilden, signifikante Werte haben, 
45 wobei die Zahl der Erscheinungsfrequenzen, die charakteristische sind, einander annahern und die Zahl 

kleiner als ein Schwellenwert ist. 

19. Farbbilderkennungsverfahren nach Anspruch 18, dadurch gekennzeichnet, dass: das Verfahren weiter den 
Schritt umfasst, wonach 

50 

k) eine vorbestimmte Zahl dem kieinen Bereich gegeben wird, wobei der Schritt j) verhindert hat, das der 
Schritt ct) fur den kieinen Bereich ausgefuhrt wird. 

20. Farbbilderkennungsverfahren nach Anspruch 8, dadurch gekennzeichnet, dass: das Verfahren weiter den Schritt 
55 umfasst, wonach 

I) der Schritt cl) an einer Ausfuhrung gehindert wird, falls die Charakteristiken einer jeden Referenz-Charakte- 
ristiksammiung unter der Vielzahl von Referenz-Charakteristiksammlungen eine Ahnlichkeit zu den Charak- 
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teristiken hat, die von jedem der kleinen Bereich extrahiert werden, wobei die Ahnlichkeit eine solche ist, die 
kleiner als ein vorbestimmter Grad bzw. vorbestimmter Umfang ist; und 

m) eine vorbestimmte Anzahl des kleinen Bereichs gegeben wird, wobei der Schritt I) verhindert hat, dass der 
Schritt ci) fur den kleinen Bereich ausgefuhrt wird. 

21. Farbbilderkennungsverfahren nach Anspruch 8, dadurch gekennzeichnet, dass: der Schritt c2) das Objekt aus 
der Vlelzahl von Objekten derartig auswahlt, dass: 
das somit ausgewahite Objekt vorab in kleine Bereiche unterteilt wird; 

wobei die Identifikationsnummer einer Referenz-Charakteristiksamm 

rakteristiksammlungen vorab einem jeden der kleinen Bereiche des somit ausgewahlten Objekts zugewiesen 
wird : 

wobei ein Histogramm eines Satzes von Identifikationsnummern, die zu den kleinen Bereichen des somit 
ausgewahlten Objekts zugewiesen wurden, den hochsten Grad von Ahnlichkeit zu einem Histogramm eines 
Satzes der Identifikationsnummern haben, die den somit ausgewahlten Objekt entsprechen: 
wobei jedes der Histogramme ein Histogramm von Erscheinungshaufigkeiten aufweist, 
wobei die Erscheinungshaufigkeiten eine Erscheinungshaufigkeit der kleinen Bereiche sind, die dieselbe Iden- 
tifikationsnummer aufweisen. 

20 22. Farbbilderkennungsapparat, dadurch gekennzeichnet, dass: der Apparat folgendes umfasst: 

eine Einrichtung (37) zum Extrahieren von Charakteristiken von dem Eingangs-Farbbildsignal; 
eine Einrichtung (22), urn einen Kandidatenbildteil zu finden, der einer Vielzahl von moglichen Objekten von 
einem Eingangs-Farbbild entspricht, das durch das Eingangs-Farbbildsignal dargestellt wird, und zwar parallel 
25 zu der Extrahiereinrichtung mittels einer Objekt-Extrahiereinrichtung; und 

eine Einrichtung (41), urn eines der Vielzahl von moglichen Objekten als relevant auszuwahlen, indem die 
extrahierten Charakteristiken verwendet werden, die dem Kandidatenbildteil entsprechen. 

23. Farbbilderkennungsapparat nach Anspruch 22, dadurch gekennzeichnet, dass: die Einrichtung (22) zum Finden 
30 eines Kandidatenbildteils folgendes umfasst: 

eine Einrichtung (2), urn ein umschreibendes Rechteck von einem Eingangs-Zweiton-Bildsignal zu bestimmen; 
wobei das umschreibende Rechteck ein Rechteck ist, das ein Bildteil umschreibt, das aus kontinuieriichen 
Pixeln besteht, wobei jedes Pixel ein Pixel eines vorbestimmten gleichen Tons ist; 
35 eine Einrichtung (2), urn Kontaktpositionen zu detektieren, bei denen das umschreibende Rechteck in Kontakt 

mit dem Bildteil kommt; und 

eine Einrichtung (4), urn zu bestimmen, ob Oder ob nicht das Bildteil mit einem gegebenen Objekt identisch 
ist, wobei die Bestimmung durchgefuhrt wird, indem die Kontaktposrtion verwendet werden. 

40 24. Farbbilderkennungsapparat nach Anspruch 22, dadurch gekennzeichnet, dass: die Einrichtung zum Extrahieren 
folgendes umfasst: 

eine Einrichtung (36), urn ein Eingangs-Farbbild in kleine Bereiche zu unterteilen; 
eine Einrichtung (37), urn Charakteristiken von den kleinen Bereichen zu extrahieren: 

45 

wobei die Einrichtung zum Auswahlen folgendes umfasst: 

eine Einrichtung (38), urn eine Referenz-Charakteristiksammlung aus einer Vielzahl von Referenz-Charakte- 
ristiksammlungen fur jede der kleinen Bereiche auszuwahlen, so dass die somit ausgewahite Referenz-Cha- 
50 rakteristiksammlung Charakteristiken aufweist, die den hochsten Grad an Ahnlichkeit mit den Charakteristiken 

aufweisen, die von jedem der kleinen Bereiche ausgewahit worden sind; 

wobei jede der Vielzahl von Ref erenz-Charakteristiksammlungen eine identifikationsnummer aufweisen und somit 
die Identifikationsnummer der somit ausgewahlten Referenz-Charakteristiksammlung zu jedem der kleinen Berei- 
55 che zugewiesen wird; und 

eine Einrichtung (41), urn ein Objekt aus einer Vielzahl von Objekten als relevant auszuwahlen, so dass 
das somit ausgewahite Objekt vorab in kleine Bereiche unterteilt worden ist, 
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dieldentifikationsnummereiner Referenz-Charakteristiksammlung unterder Vielzahi von Referenz-Charakte- 
ristiksammlungen vorab zu jedem der kleinen Bereiche des somit ausgewahlten Objekts zugewiesen wurde, 
Charakteristiken eines Satzes der Identifikationsnumrnern, die den kleinen Bereichen des somit ausgewahlten 
Objekts zugewiesen wurden, den hochsten Grad an Ahnlichkeit mit Charakteristiken eines Satzes von den 
Identifikationsnummern haben, die dem somit ausgewahlten Objekt entsprechen. 

Revendications 

1 . Procede de reconnaissance d'images en couleurs, caracterise en ce que: 
ledit procede comprend les etapes consistant a: 

a) (S202A, S202B) extraire des caracteristiques d'un signal d'image en couleurs d'entree; 

b) (S1 01 a S1 08) trouver une partie d'image candidate correspondant a une pluralite d'objets possibles a partir 
d'une image en couleur d'entree representee par ledit signal d'image en couleurs d'entree, en parallele avec 
I'etape a) a I'aide d'un procede d'extraction d'objets; et 

c) (S205) selectionner run de ladite pluralite d'objets possibles comme important en utilisant lesdites carac- 
teristiques extraites, qui correspondent a la partie d'image candidate. 

2. Procede de reconnaissance d'images en couleurs selon la revendication 1 , caracterise en ce que: 

ladite etape c) est executee uniquement apres I'instant ou ladite etape b) a trouve ladite partie d'image can- 
didate. 

3. Procede de reconnaissance d'images en couleurs selon la revendication 1 ou 2, selon lequel ladite 6tape b) inclut 
les etapes consistant a: 

ba) (S102) determiner un rectangle d'encadrement a partir d'un signal d'image d'entree a deux teintes; ledit 
rectangle d'encadrement etant un rectangle entourant une partie de 1'image qui comprend des pixels continus, 
chacun desdits pixels etant un pixel d'une meme teinte predetermined; 

(bb) (S102) detecter des positions de contact au niveau desquelles ledit rectangle d'encadrement vient en 
contact avec ladite partie d'image; et 

(be) (S1 06 : S1 07, S 1 08, S11 5, S11 6) determiner si ladite partie d'image est identique ou non a un objet donne, 
la determination etant executee en utilisant lesdites positions de contact. 

4. Procede de reconnaissance d'images en couleurs selon la revendication 3, caracterise en ce que: 

ledit objet donne possede une forme rectangulaire, ladite forme rectangulaire dudit objet donne possedant 
des cdtes ayant des longueurs predeterminees; et 
ladite etape be) comprend les etapes consistant a: 

bc-1 ) (S1 06, S1 07) determiner sur la base d'une hypothese selon laqueile ladite partie d'image possede 
une forme rectangulaire, la longueur des cotes de ladite forme rectangulaire de ladite partie d'image; 
bc-2) (S1 08) comparer lesdites longueurs de cotes de ladite forme rectangulaire de ladite partie d'image 
auxdites longueurs predeterminees de cotes de ladite forme rectangulaire desdits objets, et 
bc-3) (S11 5, S11 6) determiner si ladite partie d'image est identique ou non audit objet donne, sur la base 
d'un resultat de la comparaison executee avec ladite etape bc-2). 

r.. Procede de reconnaissance d'images en couleurs selon la revendication 4, caracterise en ce que: 

ladite etape b) comprend en outre une etape consistant a 

bd) (S105) determiner si ladite partie d'image est ou non une partie d'image obtenue en tant que resultat de 
la rotation de ladite forme rectangulaire dudit objet; et selon lequel: 

ladite etape bc-1) peut determiner correctement des longueurs de cote de ladite forme rectangulaire de 
la partie d'image en utilisant un resultat de determination obtenu lors de ladite etape bd). 

6. Procede de reconnaissance d'images en couleurs selon la revendication 3, caracterise en ce que: 
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ladite etape b) comprend en outre les etapes consistant a: 

be-1 ) (S1 01 ) obtenir un premier signal parmi lesdits signaux d'image d'entree a deux teintes a partir d'un 
signal d'image en couleurs d'entree, d'une premiere maniere, en tant que resultat de la comparaison de 
!a valeur de luminosite representee par (edit signal d'image en couleurs d'entree a une premiere valeur 
de seuil; 

be-2) (S101) obtenir un second signal parmi lesdits signaux d'image d'entree a deux teintes a partir dudit 
signal d'image en couleurs d'entree, d'une seconde maniere, en tant que resultat de la comparaison de 
ladite valeur de luminosite representee par ledit signal d'image en couleurs d'entree a une premiere valeur 
de seuil; et 

selon lequel ladite etape ba) determine un premier rectangle parmi lesdits rectangles d'encadrement a 
partir dudit premier signal parmi lesdits signaux d'images d'entree a deux teintes, et determiner un second 
rectangle parmi lesdits rectangles d'encadrement a partir dudit second signal parmi lesdits signaux d'ima- 
ges d'entree a deux teintes. 

Procede d'extraction d'objets selon la revendication 6, caracterise en ce que: 
ladite etape b) comprend en outre les etapes consistant a: 

bf) determiner si ledit premier parmi lesdits rectangles d'encadrement inclut completement ou non du point 
de vue geometrique ledit second rectangle parmi lesdits rectangles d'encadrement, et determiner si ledit se- 
cond rectangle parmi lesdits rectangles d'encadrement inclut completement ou non du point de vue geome- 
trique ledit premier rectangle parmi lesdits rectangles d'encadrement; et 

bg) eliminer I'un desdits premier et second rectangles d'encadrement a partir de la mise en oeuvre desdites 
etapes bb) et be), si ladite etape bf) a etabli que ledit rectangle d'encadrement inclut completement, du point 
de vue geometrique, I'un restant desdits premier et second rectangles d'encadrement. 

Procede de reconnaissance d'images en couleurs selon la revendication 1 , caracterise en ce que: 
une etape a) comprend deux etapes consistant a 

a1 ) (S201 ) diviser une image en couleurs d'entree en de petites regions; 

a2) (S202A) extraire des caracteristiques a partir desdites petites regions dudit signal d'image en couleurs 
d'entree, 

une etape c) comprend deux etapes consistant a: 

c1) (S204) selectionner un ensemble de caracteristiques de reference par une pluraiite d'ensembles de ca- 
racteristiques de reference pourchacune desdites petites regions de telle sorte que ledit ensemble de carac- 
teristiques de reference ainsi selectionne possede des caracteristiques possedant le degre de similitude maxi- 
mum pour iesdites caracteristiques extraites de chacune desdites petites regions; 

chacun de ladite pluraiite d'ensembles de caracteristiques de reference possede un numero d'identification 
et par consequent le numero d'identification de I'ensemble de caracteristiques de reference ainsi selectionne 
est fourni pour chacune desdites petites regions; et 

c2) (S205) selectionner un objet parmi ladite pluraiite d'objets possibles comme etant important de telle sorte 
que: 

I'objet ainsi selectionne est divise prealablement en les petites regions, 

le numero d'identification d'un ensemble de caracteristiques de reference de ladite pluraiite d'ensemble 
de caracteristiques de reference est donne prealablement a chacune desdites petites regions dudit objet 
ainsi selectionne, et 

des caracteristiques d'un ensemble des numeros d'identification prevus pour iesdites petites regions dudit 
objet ainsi selectionne possedent le degre maximum de similitude avec des caracteristiques d'un ensem- 
ble des numeros d'identification correspondant a I'objet ainsi selectionne. 

Procede de reconnaissance d'images en couleurs selon la revendication 8, caracterise en ce que: 
ladite etape a2) comprend les etapes consistant a: 

a2-1) echantillonner des pixels parmi des pixels constituant chacune desdites petites regions qui constituent 
I'image en couleurs d'entree, et 

b-2) utiliser des donnees d'images des pixels ainsi echantillonnes pour extraire Iesdites caracteristiques des- 
dites petites regions. 
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10. Precede de reconnaissance d'images en couleurs selon la revendication 8, caracterise en ce que: 
ledit precede comprend en outre une etape consistant a: 

e) convertir lesdites caracteristiques, qui ont ete extraites par ladite etapes a2). de telle sorte qu'une valeur 
de date desdites caracteristiques est reduite. 

11. Precede de reconnaissance d'images selon la revendication 8, caracterise en ce que: 
ledit procede comprend en outre les etapes consistant a: 

f) obtenir une valeur pour chacun de ladite pluralite d'objets, ladite valeur representant la maniere dont les 
ensembles de caracteristiques de reference donnes auxdites petites regions de chacun de ladite pluralite 
d'objets estsimilaire aux ensembles de caracteristiques de reference donnes auxdites petites regions de ladite 
image en couleurs d'entree; et 

g) comparer ladite valeur obtenue au moyen de I'etape f) a une valeur de seuii de maniere a determiner si 
chacun de ladite pluralite d'objets est un candidat d'objets parmi ladite pluralite d'objets, ledit candidat d'objets 
etant forme par les objets qui presentent une possibility d'etre determines comme identiques a.ladite imaqe 
en couleurs d'entree. 

12. Procede de reconnaissance d'images en couleurs selon la revendication 11 , caracterise en ce que ladite valeur 
de seuil utilisee lors de ladite etape g) est determinee individuellement pour chacun de ladite pluralite d'objets sur 
la base de caracteristiques de ladite pluralite d'objets. 

13. Procede de reconnaissance d'images en couleurs selon la revendication 8, selon lequel un mode intervenant lors 
de ladite etape d) varie entre lesdites petites regions constituant ladite image en couleurs d'entree, en fonction 
des caracteristiques extraites au moyen de ladite etape a2) pour chacune desdites petites regions constituant 
ladite image en couleurs d'entree. 

14. Procede de connaissance d'images en couleur selon la revendication 8, caracterise en ce que: 

chacun des ensembles de caracteristiques comprend un histogramme de frequences d'apparition, les numeros 
^identification d'ensembles de caracteristiques parmi lesdits ensembles de caracteristiques etant donnes aux 
petites regions de ladite image en couleurs d'entree et a ladite pluralite d'objets.. chacune desdites frequences 
d'apparition etant une frequence d'apparition de pixels ayant les memes caracteristiques. 

15. Procede de reconnaissance d'images en couleurs selon la revendication 14. caracterise en ce que 
lesdites caracteristiques extraites lors de ladite etape a2) comprennent des chromaticites. 

16. Procede de reconnaissance d'images en couleurs seion la revendication 14 : caracterise en ce que: 
ledit procede comprend en outre une etape consistant a 

h) empecher I'execution de ladite etape d ), si la frequence d'apparition de I'histogramme d'une petite region 
parmi lesdites petites regions constituant ladite image en couleurs d'entree depasse une vaieur predeterminee. 

17. Procede de reconnaissance d'images en couleurs selon la revendication 16, caracterise en ce que: 
iedit procede comprend en outre une etape consistant a 

i) donner un numero predetermine a ladite petite region, dont ladite frequence d'apparition depasse ladite 
valeur predeterminee lors de ladite etape h). 

18. Procede de reconnaissance d'images en couleurs selon la revendication 14, caracterise en ce que: 
ledit procede comprend en outre une etape consistant a 

j) empecher I'execution de ladite etape d ) si seui un nombre de frequences d'apparition parmi les frequences 
d'apparition constituant i'histogramme d'une petite region parmi lesdites petites regions constituant fadite ima- 
ge en couleurs d'entree possedent des valeurs significatives, 

ledit nombre de frequences d'apparition etant les frequences des caracteristiques qui sont voisines les unes 
des autres, lesdits nombres etant inferieurs a ladite valeur de seuil. 

19. Precede de reconnaissance d'images en couleur selon la revendication 18, caracterise en ce que: 
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ledit precede comprend en outre une etape consistant a: 

k) donner un numero predetermine a ladite petite region, iadite etape j) ayant empecher I'execution de ladite 
etape c1 ) pour ladite petite region. 

5 

20. Procede de reconnaissance d'images en couteurs selon la revendication 8, caracterise en ce que: 
ledit procede comprend en outre une etape consistant a: 

I) empecher I'execution de ladite etape c1) 
10 si les caracteristiques de chaque ensemble de caracteristiques de reference parmi ladite pluralite d'ensembles 

de caracteristiques de reference presentent des similitudes avec lesdites caracteristiques extraites de chacune 
desdites petites regions, ladite similitude etant inferieure a un degre predetermine; et 
m) donner un numero predetermine a ladite petite region, ladite etape 1 ) ayant empecher I'execution de ladite 
etape d ) pour ladite petite region. 

15 

21 . Procede de reconnaissance d'images en couleurs selon la revendication 8, caracterise en ce que: 
ladite etape c2) selectionne I'objet parmi ladite pluralite d'objets de telle sorte que: 

I'objet ainsi selectionne est divise prealablement en les petites regions; 
20 le numero d'identification d'un ensemble de caracteristiques de reference parmi ladite pluralite d'ensembles 

de caracteristiques de reference ayant ete prealablement donne a chacune desdites petites regions dudit 
objet ainsi selectionne, 

un histogramme d'un ensemble des numeros d'identification donnes auxdites petites regions dudit objet ainsi 
selectionne possedant le degre maximum de similitude avec un histogramme d'un ensemble des numeros 
25 d'identification correspondant a i'objet ainsi selectionne; 

chacun des histogrammes comprenant un histogramme de frequences d'apparition, chacune desdites fre- 
quences d'apparition etant une frequence d'apparition des petites regions et ayant le meme numero d'identi- 
fication. 

30 22. Dispositif de reconnaissance d'images en couleurs caracterise en ce que: 
ledit dispositif comprend: 

des moyens (37) pour extraire des caracteristiques d'un signal d'image en couleurs d'entree; 
des moyens (22) pourtrouver une partie d'image candidate correspondant a une pluralite d'objets possibles 
35 a partir d'une image en couleurs d'entree representee par ledit signal d'image en couleurs d'entree, en parallele 

avec lesdits moyens d'extraction a I'aide de moyens d'extraction d'objets; et 

des moyens (41) pour selectionner I'un de ladite pluralite d'objets possibles comme important moyennant 
I'utilisation desdites caracteristiques extraites qui correspondent a la partie d'image candidate. . 

40 23. Dispositif de reconnaissance d'image en couleurs selon la revendication 22, caracterise en ce que: 
lesdits moyens (22) pour fixer une partie d'image candidate comprennent: 

des moyens (2) pour determiner un rectangle d'encadrement a partir d'un signal d'image d'entree a deux 
teintes; 

45 ledit rectangle d'encadrement etant un rectangle encadrant une partie d'image qui est constitute de pixels 

continus, chacun desdits pixels etant un pixel ayant une meme teinte predeterminee; 
des moyens (2) pour detecter des positions de contact, dans lesquelles ledit rectangle d'encadrement vient 
en contact avec ladite partie d'image; et 

des moyens (4) pour determiner si ladite partie d'image est identique ou non a un objet donne, ladite deter- 
50 mination etant executee en utilisant lesdites parties de contact. 

24. Dispositif de reconnaissance d'images en couleurs selon la revendication 22, caracterise en ce que: 
lesdits moyens d'extraction comprennent: 

55 des moyens (36) pour diviser une image d'entree en couleurs en petites regions; 

des moyens (37) pour extraire des caracteristiques a partir desdites petites regions; 
lesdits moyens de selection comprennent: 



31 



EP 0 657 839 B1 



des moyens (38) pour selectionner un ensemble de caracteristiques de reference parmi une pluralite 
d'ensembles de caracteristiques de reference pour chacune desdites petites regions, de telle sorte que 
Pensemble ainsi selectionne de caracteristiques de reference possede des caracteristiques presentant le 
degre maximal de similitude avec lesdites caracteristiques extraites de chacune desdites petites regions; 
chacun de ladite pluralite d'ensembles de caracteristiques de reference possedant un numero d'identifi- 
cation, et ainsi le numero d'identification de ('ensemble ainsi selectionne de caracteristiques de reference 
est donne a chacune desdites petites regions; et 

des moyens (41) pour selectionner un objet parmi ladite pluralite d'objets en tant qu'important, de telle 
sorte que: 

I'objet ainsi selectionne etant prealablement divise en les petites regions, 

le numero d'identification d'un ensemble de caracteristiques de reference parmi ladite pluralite d'en- 
sembles de caracteristiques de reference etant precedemment fixe pour chacune desdites petites 
regions dudit objet ainsi selectionne, 

des caracteristiques d'un ensemble des numeros d'identification donne pour lesdites petites regions 
dudit objet ainsi selectionne possedant le degre maximum de similitude avec des caracteristiques 
d'un ensemble des numeros d'identification correspondant a I'objet ainsi selectionne. 
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